vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #299
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.QpiJr1R5gD', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13(finished: start all VMs, in 66.94 seconds)14gitlab # Discarding device blocks: 0/262144 done15gitlab: waiting for unit setup-gitlab.service16gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes17gitlab: waiting for the VM to finish booting18gitlab # Filesystem UUID: 0445db83-2b37-41ae-9a75-ec8804b2007519gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1sjj42kipx9cady32f8vkacjdarwgaqg-nixos-system-gitlab-test/init regInfo=/nix/store/7cy9i34yp8r0wdj4fwl6v4v8kh1c123g-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000001] kvm-clock: using sched offset of 519747358 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor67gitlab # [ 0.000814] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000881] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002734] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002745] Using GB pages for direct mapping73gitlab # [ 0.002796] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002805] ACPI: Early table checksum verification disabled75gitlab # [ 0.002807] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002811] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002814] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002821] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002823] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002825] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002827] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002828] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002830] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002831] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002831] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002832] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002832] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002833] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003060] No NUMA configuration found90gitlab # [ 0.003061] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003063] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]92gitlab # [ 0.005323] Zone ranges:93gitlab # [ 0.005324] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005326] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005327] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005328] Device empty97gitlab # [ 0.005329] Movable zone start for each node98gitlab # [ 0.005329] Early memory node ranges99gitlab # [ 0.005330] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005331] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005332] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005333] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005351] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005619] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.113706] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.114139] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.114154] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.114185] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.114187] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.114189] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.114190] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.114191] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.114192] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.114194] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.114195] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.114201] TSC deadline timer available117gitlab # [ 0.114205] CPU topo: Max. logical packages: 1118gitlab # [ 0.114206] CPU topo: Max. logical dies: 1119gitlab # [ 0.114207] CPU topo: Max. dies per package: 1120gitlab # [ 0.114210] CPU topo: Max. threads per core: 1121gitlab # [ 0.114211] CPU topo: Num. cores per package: 4122gitlab # [ 0.114211] CPU topo: Num. threads per package: 4123gitlab # [ 0.114211] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.114236] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.114253] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.114255] kvm-guest: setup PV sched yield127gitlab # [ 0.114271] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.114272] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.114273] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.114274] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.114276] Booting paravirtualized kernel on KVM132gitlab # [ 0.114280] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.118768] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.120969] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.121021] kvm-guest: PV spinlocks enabled136gitlab # [ 0.121023] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.121026] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1sjj42kipx9cady32f8vkacjdarwgaqg-nixos-system-gitlab-test/init regInfo=/nix/store/7cy9i34yp8r0wdj4fwl6v4v8kh1c123g-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.121120] Unknown kernel command line parameters "regInfo=/nix/store/7cy9i34yp8r0wdj4fwl6v4v8kh1c123g-closure-info/registration", will be passed to user space.139gitlab # [ 0.121133] random: crng init done140gitlab # [ 0.121134] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.129919] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.134177] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.134253] software IO TLB: area num 4.144gitlab # [ 0.208224] Fallback order for Node 0: 0145gitlab # [ 0.208231] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.208232] Policy zone: Normal147gitlab # [ 0.210725] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.220270] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.233104] allocated 12582912 bytes of page_ext150gitlab # [ 0.243181] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.243184] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.244014] Dynamic Preempt: lazy153gitlab # [ 0.244175] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.244176] rcu: RCU event tracing is enabled.155gitlab # [ 0.244176] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.244178] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.244178] Rude variant of Tasks RCU enabled.158gitlab # [ 0.244179] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.244179] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.244180] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.244204] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.244206] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.244207] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.248544] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.248829] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.248836] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.248948] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.252513] Console: colour VGA+ 80x25169gitlab # [ 0.252516] printk: legacy console [tty0] enabled170gitlab # [ 0.283029] printk: legacy console [ttyS0] enabled171gitlab # [ 0.392180] ACPI: Core revision 20250807172gitlab # [ 0.393131] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.394734] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.395866] x2apic enabled175gitlab # [ 0.396663] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.397597] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.398864] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.400336] kvm-guest: setup PV IPIs179gitlab # [ 0.401920] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.402977] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns181gitlab # [ 0.404777] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)182gitlab # [ 0.405852] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.406922] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.408775] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.409779] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.411775] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.412775] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.414607] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.415775] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.416774] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.417783] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.418775] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.419775] active return thunk: srso_alias_return_thunk194gitlab # [ 0.421481] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.422773] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.423774] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.424775] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.425775] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.426774] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.427775] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.428775] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.430579] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.431587] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.432775] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.433774] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.434774] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.464389] Freeing SMP alternatives memory: 44K208gitlab # [ 0.464778] pid_max: default: 32768 minimum: 301209gitlab # [ 0.465883] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.466870] landlock: Up and running.211gitlab # [ 0.467775] Yama: becoming mindful.212gitlab # [ 0.468918] LSM support for eBPF active213gitlab # [ 0.469839] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.470911] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.473773] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.474331] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.474780] ... version: 2218gitlab # [ 0.475521] ... bit width: 48219gitlab # [ 0.475777] ... generic counters: 6220gitlab # [ 0.476504] ... generic bitmap: 000000000000003f221gitlab # [ 0.476788] ... fixed-purpose counters: 0222gitlab # [ 0.477536] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.477777] ... value mask: 0000ffffffffffff224gitlab # [ 0.478715] ... max period: 00007fffffffffff225gitlab # [ 0.479502] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.479894] signal: max sigframe size: 3376227gitlab # [ 0.480702] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.481399] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.481954] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.487379] smp: Bringing up secondary CPUs ...231gitlab # [ 0.488174] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.488780] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.491942] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.493489] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)235gitlab # [ 0.494077] Memory: 6011300K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved)236gitlab # [ 0.495777] devtmpfs: initialized237gitlab # [ 0.496633] x86/mm: Memory block size: 128MB238gitlab # [ 0.499863] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.500849] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.501877] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.503008] PM: RTC time: 10:13:00, date: 2026-09-03242gitlab # [ 0.506617] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.509051] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.509836] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.511858] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.512788] audit: initializing netlink subsys (disabled)247gitlab # [ 0.513835] audit: type=2000 audit(1788430380.519:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.514076] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.515778] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.516777] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.517757] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.518777] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.519793] cpuidle: using governor menu254gitlab # [ 0.521983] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.523025] PCI: Using configuration type 1 for base access256gitlab # [ 0.523778] PCI: Using configuration type 1 for extended access257gitlab # [ 0.524919] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.526828] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.527778] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.528778] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.529777] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.534967] ACPI: Added _OSI(Module Device)263gitlab # [ 0.536785] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.538785] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.545915] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.550160] ACPI: Interpreter enabled267gitlab # [ 0.551890] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.552780] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.553960] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.555779] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.557022] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.563785] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.564786] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.567353] acpiphp: Slot [3] registered275gitlab # [ 0.567827] acpiphp: Slot [4] registered276gitlab # [ 0.568689] acpiphp: Slot [5] registered277gitlab # [ 0.570412] acpiphp: Slot [6] registered278gitlab # [ 0.570812] acpiphp: Slot [7] registered279gitlab # [ 0.571678] acpiphp: Slot [8] registered280gitlab # [ 0.572406] acpiphp: Slot [9] registered281gitlab # [ 0.573811] acpiphp: Slot [10] registered282gitlab # [ 0.574681] acpiphp: Slot [11] registered283gitlab # [ 0.575428] acpiphp: Slot [12] registered284gitlab # [ 0.575813] acpiphp: Slot [13] registered285gitlab # [ 0.576675] acpiphp: Slot [14] registered286gitlab # [ 0.577425] acpiphp: Slot [15] registered287gitlab # [ 0.578818] acpiphp: Slot [16] registered288gitlab # [ 0.579692] acpiphp: Slot [17] registered289gitlab # [ 0.580461] acpiphp: Slot [18] registered290gitlab # [ 0.580820] acpiphp: Slot [19] registered291gitlab # [ 0.581682] acpiphp: Slot [20] registered292gitlab # [ 0.583407] acpiphp: Slot [21] registered293gitlab # [ 0.583807] acpiphp: Slot [22] registered294gitlab # [ 0.584688] acpiphp: Slot [23] registered295gitlab # [ 0.585419] acpiphp: Slot [24] registered296gitlab # [ 0.586807] acpiphp: Slot [25] registered297gitlab # [ 0.587672] acpiphp: Slot [26] registered298gitlab # [ 0.588441] acpiphp: Slot [27] registered299gitlab # [ 0.588813] acpiphp: Slot [28] registered300gitlab # [ 0.589659] acpiphp: Slot [29] registered301gitlab # [ 0.590408] acpiphp: Slot [30] registered302gitlab # [ 0.591818] acpiphp: Slot [31] registered303gitlab # [ 0.592669] PCI host bridge to bus 0000:00304gitlab # [ 0.593406] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.594779] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.595780] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.598783] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.599778] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.600778] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.601968] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.604261] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.606204] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.609360] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.609805] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.610777] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.611779] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.612777] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.613960] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.616805] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.618896] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.620045] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.621784] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.622991] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.635841] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.636790] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.637787] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.638945] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.641516] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.645785] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.646784] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.647799] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.660755] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.662961] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.666791] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.667669] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.668563] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.671009] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.675785] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.676783] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.677793] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.681011] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.683659] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.684459] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.685799] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.689164] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.692711] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.693467] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.693799] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.697001] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.700785] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.701692] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.702568] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.704975] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.707785] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.710461] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.710799] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.711797] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.713991] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.716789] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.717876] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.725704] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.736801] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.738799] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.740837] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.747862] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.756800] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.758799] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.761837] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.774000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.776087] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.778043] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.780039] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.782939] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.790092] iommu: Default domain type: Translated375gitlab # [ 0.791784] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.796135] ACPI: bus type USB registered377gitlab # [ 0.798037] usbcore: registered new interface driver usbfs378gitlab # [ 0.801005] usbcore: registered new interface driver hub379gitlab # [ 0.802811] usbcore: registered new device driver usb380gitlab # [ 0.808879] NetLabel: Initializing381gitlab # [ 0.809783] NetLabel: domain hash size = 128382gitlab # [ 0.811783] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.814990] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.816804] PCI: Using ACPI for IRQ routing385gitlab # [ 0.819937] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.820773] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.820773] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.828794] vgaarb: loaded389gitlab # [ 0.830296] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.832782] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.841942] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.846481] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.847890] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.849802] pnp: PnP ACPI init395gitlab # [ 0.852676] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.866480] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.870178] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.872681] NET: Registered PF_INET protocol family399gitlab # [ 0.877800] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.901509] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.902998] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.904407] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.907981] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.909344] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.910540] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.911877] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.913075] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.914424] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.915403] NET: Registered PF_XDP protocol family410gitlab # [ 0.916266] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.917308] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.918347] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.919486] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.920630] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.921990] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.922985] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.925361] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.927830] PCI: CLS 0 bytes, default 64419gitlab # [ 0.928613] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.929133] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.929691] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.932437] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns423gitlab # [ 0.953130] Initialise system trusted keyrings424gitlab # [ 0.954278] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 0.965720] Key type asymmetric registered426gitlab # [ 0.966472] Asymmetric key parser 'x509' registered427gitlab # [ 0.967381] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 0.969185] io scheduler mq-deadline registered429gitlab # [ 0.971160] io scheduler kyber registered430gitlab # [ 0.977374] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 0.978690] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 0.981100] Linux agpgart interface v0.103433gitlab # [ 0.981870] ACPI: bus type drm_connector registered434gitlab # [ 0.984308] usbcore: registered new interface driver usbserial_generic435gitlab # [ 0.985412] usbserial: USB Serial support registered for generic436gitlab # [ 0.986415] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 0.986415] Please enable it if your BIOS has the CPPC option.438gitlab # [ 0.988713] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 0.990124] drop_monitor: Initializing network drop monitor service440gitlab # [ 0.991334] NET: Registered PF_INET6 protocol family441gitlab # [ 0.993101] Segment Routing with IPv6442gitlab # [ 0.993791] In-situ OAM (IOAM) with IPv6443gitlab # [ 0.996232] IPI shorthand broadcast: enabled444gitlab # [ 1.000535] sched_clock: Marking stable (852011452, 147937490)->(1021194425, -21245483)445gitlab # [ 1.002354] registered taskstats version 1446gitlab # [ 1.003377] Loading compiled-in X.509 certificates447gitlab # [ 1.013137] Demotion targets for Node 0: null448gitlab # [ 1.014368] Key type .fscrypt registered449gitlab # [ 1.015091] Key type fscrypt-provisioning registered450gitlab # [ 1.016162] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.017110] ima: Allocated hash algorithm: sha1452gitlab # [ 1.017955] ima: No architecture policies found453gitlab # [ 1.019080] PM: Magic number: 10:470:223454gitlab # [ 1.020776] RAS: Correctable Errors collector initialized.455gitlab # [ 1.025344] clk: Disabling unused clocks456gitlab # [ 1.026064] PM: genpd: Disabling unused power domains457gitlab # [ 1.214626] Freeing initrd memory: 29392K458gitlab # [ 1.218206] Freeing unused decrypted memory: 2028K459gitlab # [ 1.220890] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.222007] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.224069] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.225701] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.277093] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.278189] Run /init as init process465gitlab # [ 1.289276] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.310276] fuse: init (API version 7.45)467gitlab # [ 1.318926] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.325141] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.328169] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.369485] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.706349] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.746458] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.748182] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.749885] systemd[1]: Running in initrd.475gitlab # [ 1.751850] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.754158] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 1.855083] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 1.917611] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 1.927684] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 1.928854] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 1.930284] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 1.931464] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 1.932588] systemd[1]: Reached target Path Units.484gitlab # [ 1.933462] systemd[1]: Reached target Slice Units.485gitlab # [ 1.934355] systemd[1]: Reached target Swaps.486gitlab # [ 1.935178] systemd[1]: Reached target Timer Units.487gitlab # [ 1.936156] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 1.937379] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 1.938508] systemd[1]: Listening on Journal Sockets.490gitlab # [ 1.939521] systemd[1]: Listening on udev Control Socket.491gitlab # [ 1.940556] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 1.941532] systemd[1]: Reached target Socket Units.493gitlab # [ 1.943483] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 1.945579] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 1.947673] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 1.953025] systemd[1]: Starting Journal Service...497gitlab # [ 1.954799] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 1.955769] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 1.962021] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 1.964114] systemd[1]: Finished Create List of Static Device Nodes.501gitlab # [ 1.974261] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...502gitlab # [ 1.994073] systemd-journald[92]: Collecting audit messages is disabled.503gitlab # [ 1.995048] systemd[1]: modprobe@configfs.service: Deactivated successfully.504gitlab # [ 1.996717] netfs: FS-Cache loaded505gitlab # [ 1.999258] systemd[1]: Finished Load Kernel Module configfs.506gitlab # [ 2.000856] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config507gitlab # [ 2.005050] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.508gitlab # [ 2.007271] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev509gitlab # [ 2.011644] systemd[1]: Finished Load Kernel Modules.510gitlab # [ 2.012177] 9pnet: Installing 9P2000 support511gitlab # [ 2.014596] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.512gitlab # [ 2.017707] systemd[1]: Starting Apply Kernel Variables...513gitlab # [ 2.020431] systemd[1]: Starting Create Static Device Nodes in /dev...514gitlab # [ 2.022008] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.515gitlab # [ 2.023673] systemd[1]: Finished Load Kernel Module 9pnet_virtio.516gitlab # [ 2.030039] systemd[1]: Finished Apply Kernel Variables.517gitlab # [ 2.032224] systemd[1]: Finished Create Static Device Nodes in /dev.518gitlab # [ 2.033538] systemd[1]: Reached target Preparation for Local File Systems.519gitlab # [ 2.034813] systemd[1]: Reached target Local File Systems.520gitlab # [ 2.036863] systemd[1]: Starting Rule-based Manager for Device Events and Files...521gitlab # [ 2.048242] systemd[1]: Started Journal Service.522gitlab # [ 1.900450] systemd-modules-load[93]: Using 4 probe threads523gitlab # [ 1.902882] systemd-modules-load[93]: Inserted module 'virtio_balloon'524gitlab # [ 1.904573] systemd-modules-load[93]: Inserted module 'virtio_gpu'525gitlab # [ 1.906736] systemd-modules-load[93]: Inserted module 'dm_mod'526gitlab # [ 1.907743] systemd[1]: Starting Create System Files and Directories...527gitlab # [ 1.913106] systemd[1]: Finished Create System Files and Directories.528gitlab # [ 1.918306] systemd-udevd[104]: Using default interface naming scheme 'v261'.529gitlab # [ 1.929718] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 1.945461] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 1.946270] systemd[1]: Reached target System Initialization.532gitlab # [ 1.947112] systemd[1]: Reached target Basic System.533gitlab # [ 2.196760] SCSI subsystem initialized534gitlab # [ 2.203260] uhci_hcd 0000:00:01.2: UHCI Host Controller535gitlab # [ 2.204211] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1536gitlab # [ 2.207177] uhci_hcd 0000:00:01.2: detected 2 ports537gitlab # [ 2.209333] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140538gitlab # [ 2.210835] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18539gitlab # [ 2.212145] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1540gitlab # [ 2.214065] usb usb1: Product: UHCI Host Controller541gitlab # [ 2.214808] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd542gitlab # [ 2.215761] usb usb1: SerialNumber: 0000:00:01.2543gitlab # [ 2.220901] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12544gitlab # [ 2.223445] virtio_blk virtio4: 4/0/0 default/read/poll queues545gitlab # [ 2.224722] hub 1-0:1.0: USB hub found546gitlab # [ 2.224799] serio: i8042 KBD port at 0x60,0x64 irq 1547gitlab # [ 2.224975] hub 1-0:1.0: 2 ports detected548gitlab # [ 2.227302] serio: i8042 AUX port at 0x60,0x64 irq 12549gitlab # [ 2.235728] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)550gitlab # [ 2.239753] scsi host0: ata_piix551gitlab # [ 2.240910] virtio_blk virtio5: 4/0/0 default/read/poll queues552gitlab # [ 2.240928] scsi host1: ata_piix553gitlab # [ 2.242553] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0554gitlab # [ 2.243816] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0555gitlab # [ 2.247215] virtio_blk virtio5: [vdb] 11841768 512-byte logical blocks (6.06 GB/5.65 GiB)556gitlab # [ 2.124080] (udev-worker)[117]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.557gitlab # [ 2.126380] (udev-worker)[117]: Network interface NamePolicy= disabled on kernel command line.558gitlab # [ 2.129324] (udev-worker)[121]: Network interface NamePolicy= disabled on kernel command line.559gitlab # [ 2.131128] systemd[1]: Starting Virtual Console Setup...560gitlab # [ 2.153979] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.155640] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 2.405672] ata2: found unknown device (class 0)563gitlab # [ 2.408471] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100564gitlab # [ 2.412641] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5565gitlab # [ 2.446728] usb 1-1: new full-speed USB device number 2 using uhci_hcd566gitlab # [ 2.451598] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0567gitlab # [ 2.321700] systemd[1]: Found device /dev/disk/by-label/nixos.568gitlab # [ 2.327977] systemd[1]: Found device /dev/disk/by-label/nix-store.569gitlab # [ 2.329911] systemd[1]: Reached target Initrd Root Device.570gitlab # [ 2.331596] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...571gitlab # [ 2.486095] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray572gitlab # [ 2.375805] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks573gitlab # [ 2.527742] cdrom: Uniform CD-ROM driver Revision: 3.20574gitlab # [ 2.382553] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.575gitlab # [ 2.615896] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.618986] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.621702] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.623466] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.624977] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.659404] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.673787] usbcore: registered new interface driver usbhid582gitlab # [ 2.675914] usbhid: USB HID core driver583gitlab # [ 2.684264] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.687526] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.819153] systemd[1]: Mounting /sysroot...586gitlab # [ 3.115241] EXT4-fs (vda): mounted filesystem 0445db83-2b37-41ae-9a75-ec8804b20075 r/w with ordered data mode. Quota mode: none.587gitlab # [ 2.974063] systemd[1]: Mounted /sysroot.588gitlab # [ 2.975222] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 2.977801] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 2.979650] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 2.982170] systemd[1]: Mounting /sysroot/run...592gitlab # [ 2.983132] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 2.990582] systemd[1]: Mounting /sysroot/tmp/xchg...594gitlab # [ 2.992958] systemd[1]: Starting Mountpoints Configured in the Real Root...595gitlab # [ 3.003848] systemd[1]: Mounted /sysroot/nix/.rw-store.596gitlab # [ 3.155032] erofs (device vdb): mounted with root inode @ nid 128.597gitlab # [ 3.009128] systemd[1]: Mounted /sysroot/run.598gitlab # [ 3.013230] systemd[1]: Starting rw-sysroot-nix-store.service...599gitlab # [ 3.014875] systemd-sysroot-fstab-check[176]: /sysroot should be mounted in the initrd, will request daemon-reload.600gitlab # [ 3.162837] 9p: Installing v9fs 9p2000 file system support601gitlab # [ 3.018280] systemd[1]: Mounted /sysroot/nix/.ro-store.602gitlab # [ 3.024501] systemd[1]: Mounted /sysroot/tmp/shared.603gitlab # [ 3.025279] systemd[1]: Mounted /sysroot/tmp/xchg.604gitlab # [ 3.026020] systemd[1]: Reload requested from client PID 176 ('systemd-sysroot') (unit initrd-parse-etc.service)...605gitlab # [ 3.027620] systemd[1]: Reloading...606gitlab # [ 3.082820] systemd[1]: Reloading finished in 56 ms.607gitlab # [ 3.110137] systemd-sysroot-fstab-check[176]: Requesting initrd-fs.target/start/replace...608gitlab # [ 3.112999] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.115604] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.117810] systemd-sysroot-fstab-check[176]: Requesting swap.target/start/replace...611gitlab # [ 3.120509] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.122599] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.125074] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.127454] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.135168] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.137538] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.819248] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 3.869542] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 3.871533] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 3.873524] systemd[1]: Starting Find NixOS closure...621gitlab # [ 3.875456] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 3.906989] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 3.909679] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 3.923890] systemd[1]: Finished Find NixOS closure.625gitlab # [ 3.925805] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 3.927903] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 3.964760] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 3.966845] systemd[1]: Stopped target Basic System.629gitlab # [ 3.968714] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 3.970688] systemd[1]: Stopped target Path Units.631gitlab # [ 3.972458] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 3.976322] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 3.979689] systemd[1]: Stopped target Slice Units.634gitlab # [ 3.981555] systemd[1]: Stopped target Socket Units.635gitlab # [ 3.984332] systemd[1]: Stopped target System Initialization.636gitlab # [ 3.987592] systemd[1]: Stopped target Swaps.637gitlab # [ 3.989727] systemd[1]: Stopped target Timer Units.638gitlab # [ 3.991811] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 3.994259] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 3.996697] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 3.999519] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 4.001458] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 4.005418] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 4.007660] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 4.010224] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 4.012096] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 4.014093] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 4.015721] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 4.017808] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 4.019856] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 4.021794] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 4.023527] systemd[1]: Stopped target Local File Systems.653gitlab # [ 4.025078] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 4.026859] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 4.028841] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 4.030372] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 4.031551] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 4.032578] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 4.033335] systemd[1]: initrd-cleanup.service: Deactivated successfully.660gitlab # [ 4.034282] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.661gitlab # [ 4.035227] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.662gitlab # [ 4.036211] systemd[1]: Finished rw-sysroot-nix-store.service.663gitlab # [ 4.037066] systemd[1]: systemd-udevd.service: Deactivated successfully.664gitlab # [ 4.038047] systemd[1]: Stopped Rule-based Manager for Device Events and Files.665gitlab # [ 4.039053] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 4.040058] systemd[1]: Closed udev Control Socket.667gitlab # [ 4.040748] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 4.041522] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 4.042561] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 4.043479] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 4.044606] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 4.045662] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 4.046621] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 4.047537] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 4.048594] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 4.049337] systemd[1]: Reached target Switch Root.677gitlab # [ 4.050055] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.156774] initrd-nixos-activation-start[226]: booting system configuration /nix/store/1sjj42kipx9cady32f8vkacjdarwgaqg-nixos-system-gitlab-test679gitlab # [ 4.195411] initrd-nixos-activation-start[226]: running activation script...680gitlab # [ 4.418984] initrd-nixos-activation-start[249]: setting up /etc...681gitlab # [ 4.535386] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.537987] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.539745] systemd[1]: Starting Switch Root...684gitlab # [ 4.555841] systemd[1]: Switching root.685gitlab # [ 4.759130] systemd-journald[92]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 4.911635] NET: Registered PF_VSOCK protocol family687gitlab # [ 5.277369] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 5.287250] systemd[1]: Detected virtualization kvm.689gitlab # [ 5.289028] systemd[1]: Detected architecture x86-64.690gitlab # [ 5.290974] systemd[1]: Detected first boot.691gitlab # [ 5.293466] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.774347] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.868131] systemd[1]: Applying preset policy.694gitlab # [ 6.044770] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 6.159127] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 6.160434] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 6.162509] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 6.164574] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 6.165975] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 6.167455] systemd[1]: Created slice User and Session Slice.701gitlab # [ 6.168376] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 6.169637] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 6.170752] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 6.171473] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 6.172289] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 6.173160] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 6.173987] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 6.174837] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 6.175809] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 6.176740] systemd[1]: Reached target Path Units.711gitlab # [ 6.177511] systemd[1]: Reached target Remote File Systems.712gitlab # [ 6.178339] systemd[1]: Reached target Slice Units.713gitlab # [ 6.179093] systemd[1]: Reached target Swaps.714gitlab # [ 6.180478] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 6.182456] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 6.183806] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 6.185435] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 6.186411] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 6.188714] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 6.190068] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 6.191675] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 6.192835] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 6.194001] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 6.195414] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 6.198130] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 6.199219] systemd[1]: Listening on udev Control Socket.727gitlab # [ 6.200122] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 6.203124] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 6.205223] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 6.207209] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 6.209895] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 6.219080] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 6.221609] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 6.228666] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 6.260735] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 6.264385] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 6.267920] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 6.272764] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 6.274564] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 6.286911] systemd[1]: Starting Journal Service...741gitlab # [ 6.290464] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 6.294906] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 6.299223] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 6.302011] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 6.309808] systemd[1]: Starting Coldplug All udev Devices...746gitlab # [ 6.312759] systemd[1]: Listening on Journal Log Access Socket.747gitlab # [ 6.313890] systemd[1]: Mounted Huge Pages File System.748gitlab # [ 6.314425] loop: module loaded749gitlab # [ 6.314944] systemd[1]: Mounted POSIX Message Queue File System.750gitlab # [ 6.315379] EXT4-fs (vda): re-mounted 0445db83-2b37-41ae-9a75-ec8804b20075.751gitlab # [ 6.315565] systemd[1]: Mounted Kernel Debug File System.752gitlab # [ 6.318913] systemd[1]: Mounted Kernel Trace File System.753gitlab # [ 6.320291] systemd[1]: Finished Create List of Static Device Nodes.754gitlab # [ 6.321318] systemd-journald[320]: Collecting audit messages is enabled.755gitlab # [ 6.321566] systemd[1]: modprobe@configfs.service: Deactivated successfully.756gitlab # [ 6.324438] systemd[1]: Finished Load Kernel Module configfs.757gitlab # [ 6.327419] systemd[1]: Mounted FUSE Control File System.758gitlab # [ 6.330636] systemd[1]: Finished Load Kernel Modules.759gitlab # [ 6.184502] systemd[1]: Queued start job for default target Multi-User System.760gitlab # [ 6.186127] s[ 6.333192] systemd[1]: Started Journal Service.761gitlab # ystemd[1]: Unnecessary job was removed for /dev/vdb.762gitlab # [ 6.188274] systemd[1]: systemd-journald.service: Deactivated successfully.763gitlab # [ 6.189470] systemd-modules-load[322]: Using 3 probe threads764gitlab # [ 6.190466] systemd-modules-load[322]: Inserted module 'tls'765gitlab # [ 6.191460] systemd-modules-load[322]: Inserted module 'loop'766gitlab # [ 6.192635] systemd[1]: Finished Remount Root and Kernel File Systems.767gitlab # [ 6.196237] systemd-oomd[323]: No swap; memory pressure usage will be degraded768gitlab # [ 6.197671] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.769gitlab # [ 6.199298] systemd[1]: Listening on Disk Image Download Service Socket.770gitlab # [ 6.200435] systemd[1]: Mounting Kernel Configuration File System...771gitlab # [ 6.202291] systemd[1]: Starting Firewall...772gitlab # [ 6.203158] systemd[1]: Starting Flush Journal to Persistent Storage...773gitlab # [ 6.204056] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore774gitlab # [ 6.208070] systemd[1]: Starting Load/Save OS Random Seed...775gitlab # [ 6.359068] systemd-journald[320]: Received client request to flush runtime journal.776gitlab # [ 6.249246] systemd[1]: Starting Apply Kernel Variables...777gitlab # [ 6.251209] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...778gitlab # [ 6.253220] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os779gitlab # [ 6.255575] systemd[1]: Mounted Kernel Configuration File System.780gitlab # [ 6.256589] systemd[1]: Finished Apply Kernel Variables.781gitlab # [ 6.257480] systemd[1]: Finished Load/Save OS Random Seed.782gitlab # [ 6.258387] systemd[1]: Reached target First Boot Complete.783gitlab # [ 6.259195] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.784gitlab # [ 6.260265] systemd[1]: Starting Create Static Device Nodes in /dev...785gitlab # [ 6.261170] systemd[1]: Finished Flush Journal to Persistent Storage.786gitlab # [ 6.262050] systemd[1]: Finished Create Static Device Nodes in /dev.787gitlab # [ 6.263781] systemd[1]: Reached target Preparation for Local File Systems.788gitlab # [ 6.265838] systemd[1]: Starting Rule-based Manager for Device Events and Files...789gitlab # [ 6.290368] systemd-udevd[365]: Using default interface naming scheme 'v261'.790gitlab # [ 6.309510] systemd[1]: Finished Coldplug All udev Devices.791gitlab # [ 6.324820] systemd[1]: Started Rule-based Manager for Device Events and Files.792gitlab # [ 6.392826] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 6.405884] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 6.411823] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.795gitlab # [ 6.425273] (udev-worker)[427]: Network interface NamePolicy= disabled on kernel command line.796gitlab # [ 6.434549] systemd[1]: Finished Firewall.797gitlab # [ 6.449391] (udev-worker)[438]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.798gitlab # [ 6.451503] (udev-worker)[438]: Network interface NamePolicy= disabled on kernel command line.799gitlab # [ 6.462351] systemd[1]: Condition check resulted in Virtio network device being skipped.800gitlab # [ 6.463461] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore801gitlab # [ 6.464954] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67802gitlab # [ 6.467251] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore803gitlab # [ 6.468755] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os804gitlab # [ 6.469899] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os805gitlab # [ 6.625359] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3806gitlab # [ 6.643749] ACPI: button: Power Button [PWRF]807gitlab # [ 6.648815] rtc_cmos 00:05: RTC can wake from S4808gitlab # [ 6.650267] Floppy drive(s): fd0 is 2.88M AMI BIOS809gitlab # [ 6.658136] rtc_cmos 00:05: registered as rtc0810gitlab # [ 6.658866] rtc_cmos 00:05: setting system clock to 2026-09-03T10:13:06 UTC (1788430386)811gitlab # [ 6.660184] mousedev: PS/2 mouse device common for all mice812gitlab # [ 6.660327] systemd-journald[320]: Time jumped backwards, rotating.813gitlab # [ 6.662280] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs814gitlab # [ 6.663907] FDC 0 is a S82078B815gitlab # [ 6.664699] parport_pc 00:03: reported by Plug and Play ACPI816gitlab # [ 6.668034] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]817gitlab # [ 6.673198] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console818gitlab # [ 6.687383] Console: switching to colour dummy device 80x25819gitlab # [ 6.688146] [drm] Found bochs VGA, ID 0xb0c5.820gitlab # [ 6.688147] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.821gitlab # [ 6.692361] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic822gitlab # [ 6.693248] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0823gitlab # [ 6.706473] Console: switching to colour frame buffer device 160x50824gitlab # [ 6.707836] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device825gitlab # [ 6.722858] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5826gitlab # [ 6.723822] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6827gitlab # [ 6.731341] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4828gitlab # [ 6.735222] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0829gitlab # [ 6.739416] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD830gitlab # [ 6.634229] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 6.793166] ppdev: user-space parallel port driver832gitlab # [ 6.647205] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.833gitlab # [ 6.648442] systemd[1]: Stopped Virtual Console Setup.834gitlab # [ 6.656135] systemd[1]: Starting Virtual Console Setup...835gitlab # [ 6.660590] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.836gitlab # [ 6.661992] systemd[1]: Stopped Virtual Console Setup.837gitlab # [ 6.666613] systemd[1]: Starting Virtual Console Setup...838gitlab # [ 6.822867] kvm_amd: TSC scaling supported839gitlab # [ 6.823452] kvm_amd: Nested Virtualization enabled840gitlab # [ 6.824041] kvm_amd: Nested Paging enabled841gitlab # [ 6.824478] kvm_amd: LBR virtualization supported842gitlab # [ 6.825060] kvm_amd: Virtual VMLOAD VMSAVE supported843gitlab # [ 6.825629] kvm_amd: Virtual GIF supported844gitlab # [ 6.826175] kvm_amd: Virtual NMI enabled845gitlab # [ 6.858388] EDAC MC: Ver: 3.0.0846gitlab # [ 6.740350] systemd-vconsole-setup[495]: Configuration of first virtual console was skipped, ignoring remaining ones.847gitlab # [ 6.742926] systemd[1]: Finished Virtual Console Setup.848gitlab # [ 7.015781] systemd[1]: Mounting /run/wrappers...849gitlab # [ 7.042854] systemd[1]: Mounted /run/wrappers.850gitlab # [ 7.044541] systemd[1]: Reached target Local File Systems.851gitlab # [ 7.046394] systemd[1]: Listening on Boot Loader Control Service Socket.852gitlab # [ 7.048617] systemd[1]: Starting register-nix-paths.service...853gitlab # [ 7.050871] systemd[1]: Starting Create SUID/SGID Wrappers...854gitlab # [ 7.052892] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.855gitlab # [ 7.055788] systemd[1]: Starting Save Transient machine-id to Disk...856gitlab # [ 7.064423] systemd[1]: Starting Create System Files and Directories...857gitlab # [ 7.091553] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.858gitlab # [ 7.094268] systemd[1]: Finished Save Transient machine-id to Disk.859gitlab # [ 7.101666] systemd[1]: Finished Create System Files and Directories.860gitlab # [ 7.107634] systemd[1]: Starting Rebuild Journal Catalog...861gitlab # [ 7.109989] systemd[1]: Starting Record System Boot/Shutdown in UTMP...862gitlab # [ 7.130734] systemd[1]: Finished Record System Boot/Shutdown in UTMP.863gitlab # [ 7.158182] systemd[1]: Finished Rebuild Journal Catalog.864gitlab # [ 7.160847] systemd[1]: Starting Update is Completed...865gitlab # [ 7.173889] systemd[1]: Finished Update is Completed.866gitlab # [ 7.214653] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.867gitlab # [ 7.216917] systemd[1]: Finished Create SUID/SGID Wrappers.868gitlab # [ 7.504601] systemd[1]: Finished register-nix-paths.service.869gitlab # [ 7.507590] systemd[1]: Reached target System Initialization.870gitlab # [ 7.509931] systemd[1]: Started Discard unused filesystem blocks once a week.871gitlab # [ 7.512278] systemd[1]: Started logrotate.timer.872gitlab # [ 7.514059] systemd[1]: Started Daily Cleanup of Temporary Directories.873gitlab # [ 7.516264] systemd[1]: Reached target Timer Units.874gitlab # [ 7.517907] systemd[1]: Listening on D-Bus System Message Bus Socket.875gitlab # [ 7.520154] systemd[1]: Listening on Nix Daemon Socket.876gitlab # [ 7.530466] systemd[1]: Starting nixbot CI service socket...877gitlab # [ 7.532472] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.878gitlab # [ 7.535240] systemd[1]: Listening on nixbot CI service socket.879gitlab # [ 7.537223] systemd[1]: Reached target Socket Units.880gitlab # [ 7.539045] systemd[1]: Starting D-Bus System Message Bus...881gitlab # [ 7.588735] dbus-broker-launch[578]: Looking up NSS user entry for 'systemd-timesync'...882gitlab # [ 7.591783] dbus-broker-launch[578]: NSS returned no entry for 'systemd-timesync'883gitlab # [ 7.594471] dbus-broker-launch[578]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"884gitlab # [ 7.599583] systemd[1]: Started D-Bus System Message Bus.885gitlab # [ 7.617543] dbus-broker-launch[578]: Ready886gitlab # [ 7.621652] systemd[1]: Reached target Basic System.887gitlab # [ 7.624457] systemd[1]: Started backdoor.service.888gitlab # [ 7.627619] systemd[1]: Starting gitlab-config.service...889gitlab # [ 7.632529] systemd[1]: Starting Import lastlog data into lastlog2 database...890gitlab # [ 7.646226] systemd[1]: Starting Logrotate configuration check...891gitlab # [ 7.654393] systemd[1]: Starting Name Service Cache Daemon (nsncd)...892gitlab # [ 7.665563] systemd[1]: Starting Post-Boot Actions...893gitlab # connecting to host...894gitlab # [ 7.672060] systemd[1]: Started Reset console on configuration changes.895gitlab # [ 7.678348] logrotate[583]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.896gitlab # [ 7.680077] logrotate[583]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf897gitlab: Guest shell says: b'Spawning backdoor root shell...\n'898gitlab # [ 7.681044] logrotate[583]: note: 'monthly' overrides previously specified 'weekly'899gitlab # [ 7.682109] logrotate[583]: Reading state from file: /var/lib/logrotate.status900gitlab: connected to guest root shell901gitlab # [ 7.682981] logrotate[583]: state file /var/lib/logrotate.status does not exist902gitlab: (connecting took 8.35 seconds)903gitlab # [ 7.684388] logrotate[583]: Allocating hash table for state file, size 64 entries904gitlab: (finished: waiting for the VM to finish booting, in 8.35 seconds)905gitlab # [ 7.684948] logrotate[583]: Handling 4 logs906gitlab # [ 7.685860] logrotate[583]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed907gitlab # [ 7.686233] logrotate[583]: considering log /var/log/btmp908gitlab # [ 7.686846] logrotate[583]: Creating new state909gitlab # [ 7.687709] logrotate[583]: Now: 2026-09-03 10:13910gitlab # [ 7.688589] logrotate[583]: Last rotated at 2026-09-03 10:00911gitlab # [ 7.689464] logrotate[583]: log does not need rotating (log has already been rotated)912gitlab # [ 7.690340] logrotate[583]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed913gitlab # [ 7.690931] logrotate[583]: considering log /var/log/wtmp914gitlab # [ 7.691782] logrotate[583]: Creating new state915gitlab # [ 7.692623] logrotate[583]: Now: 2026-09-03 10:13916gitlab # [ 7.693517] logrotate[583]: Last rotated at 2026-09-03 10:00917gitlab # [ 7.694344] logrotate[583]: log does not need rotating (log has already been rotated)918gitlab # [ 7.694932] logrotate[583]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed919gitlab # [ 7.695793] logrotate[583]: switching euid from 0 to 165 and egid from 0 to 165 (pid 583)920gitlab # [ 7.696660] logrotate[583]: considering log /var/gitlab/state/log/*.log921gitlab # [ 7.697513] logrotate[583]: log /var/gitlab/state/log/*.log does not exist -- skipping922gitlab # [ 7.697576] logrotate[583]: Creating new state923gitlab # [ 7.698519] logrotate[583]: switching euid from 165 to 0 and egid from 165 to 0 (pid 583)924gitlab # [ 7.699498] logrotate[583]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed925gitlab # [ 7.700494] logrotate[583]: switching euid from 0 to 60 and egid from 0 to 60 (pid 583)926gitlab # [ 7.701400] logrotate[583]: considering log /var/log/nginx/*.log927gitlab # [ 7.702032] logrotate[583]: log /var/log/nginx/*.log does not exist -- skipping928gitlab # [ 7.702764] logrotate[583]: Creating new state929gitlab # [ 7.703359] logrotate[583]: not running postrotate script, since no logs were rotated930gitlab # [ 7.703874] logrotate[583]: switching euid from 60 to 0 and egid from 60 to 0 (pid 583)931gitlab # [ 7.704838] nsncd[584]: Sep 03 10:13:07.682 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"932gitlab # [ 7.705564] systemd[1]: Starting resolvconf update...933gitlab # [ 7.731500] systemd[1]: Started Name Service Cache Daemon (nsncd).934gitlab # [ 7.735190] systemd[1]: Finished Logrotate configuration check.935gitlab # [ 7.735948] systemd[1]: Finished Post-Boot Actions.936gitlab # [ 7.744714] systemd[1]: Finished Import lastlog data into lastlog2 database.937gitlab # [ 7.746987] systemd[1]: Reached target Host and Network Name Lookups.938gitlab # [ 7.747654] systemd[1]: Reached target User and Group Name Lookups.939gitlab # [ 7.751941] systemd[1]: Starting User Login Management...940gitlab # [ 7.776373] systemd[1]: Stopped target Host and Network Name Lookups.941gitlab # [ 7.778071] systemd[1]: Stopping Host and Network Name Lookups...942gitlab # [ 7.778881] systemd[1]: Stopped target User and Group Name Lookups.943gitlab # [ 7.779444] systemd[1]: Stopping User and Group Name Lookups...944gitlab # [ 7.779865] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...945gitlab # [ 7.781625] systemd[1]: nscd.service: Deactivated successfully.946gitlab # [ 7.782589] systemd[1]: Stopped Name Service Cache Daemon (nsncd).947gitlab # [ 7.783698] systemd[1]: nscd.service: Consumed 26ms CPU time over 127ms wall clock time, 7.4M memory peak, 5.5M read from disk.948gitlab # [ 7.787271] systemd-logind[622]: New seat seat0.949gitlab # [ 7.788033] systemd[1]: Starting Name Service Cache Daemon (nsncd)...950gitlab # [ 7.792295] systemd-logind[622]: Watching system buttons on /dev/input/event2 (Power Button)951gitlab # [ 7.792990] systemd-logind[622]: Watching system buttons on /dev/input/event4 (QEMU Virtio Keyboard)952gitlab # [ 7.793699] systemd-logind[622]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)953gitlab # [ 7.794225] systemd[1]: Started User Login Management.954gitlab # [ 7.800383] systemd[1]: Starting linger-users.service...955gitlab # [ 7.806905] nsncd[675]: Sep 03 10:13:07.795 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"956gitlab # [ 7.809075] systemd[1]: Started Name Service Cache Daemon (nsncd).957gitlab # [ 7.809800] systemd[1]: Reached target Host and Network Name Lookups.958gitlab # [ 7.810518] systemd[1]: Reached target User and Group Name Lookups.959gitlab # [ 7.811136] systemd[1]: linger-users.service: Deactivated successfully.960gitlab # [ 7.811634] systemd[1]: Finished linger-users.service.961gitlab # [ 7.817160] systemd[1]: Finished resolvconf update.962gitlab # [ 7.817800] systemd[1]: Reached target Preparation for Network.963gitlab # [ 7.820095] systemd[1]: Starting DHCP Client...964gitlab # [ 7.822130] systemd[1]: Starting Address configuration of eth1...965gitlab # [ 7.823251] systemd[1]: Starting Extra networking commands....966gitlab # [ 7.848562] network-addresses-eth1-start[701]: adding address 192.168.1.1/24... done967gitlab # [ 7.850895] network-addresses-eth1-start[701]: adding address 2001:db8:1::1/64... done968gitlab # [ 7.855076] systemd[1]: Finished Address configuration of eth1.969gitlab # [ 7.877267] systemd[1]: Finished Extra networking commands..970gitlab # [ 7.878027] systemd[1]: Reached target Network.971gitlab # [ 7.880488] systemd[1]: Starting gitlab-workhorse.service...972gitlab # [ 7.882073] systemd[1]: Starting Nginx Web Server...973gitlab # [ 7.884066] systemd[1]: Starting PostgreSQL Server...974gitlab # [ 7.885914] dhcpcd[707]: dhcpcd-10.3.2 starting975gitlab # [ 7.887704] dhcpcd[769]: dev: loaded udev976gitlab # [ 7.891263] systemd[1]: Starting Redis Server - redis-gitlab...977gitlab # [ 8.041017] 8021q: 802.1Q VLAN Support v1.8978gitlab # [ 8.041485] 8021q: adding VLAN 0 to HW filter on device eth1979gitlab # [ 7.893706] systemd[1]: Starting Permit User Sessions...980gitlab # [ 7.903055] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.981gitlab # [ 7.913823] systemd[1]: Finished Permit User Sessions.982gitlab # [ 7.915402] systemd[1]: Started Getty on tty1.983gitlab # [ 7.916169] systemd[1]: Reached target Login Prompts.984gitlab # [ 8.124842] cfg80211: Loading compiled-in X.509 certificates for regulatory database985gitlab # [ 8.131132] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'986gitlab # [ 8.131864] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'987gitlab # [ 8.133515] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2988gitlab # [ 8.134663] cfg80211: failed to load regulatory.db989gitlab # [ 8.154683] 8021q: adding VLAN 0 to HW filter on device eth0990gitlab # [ 8.008043] dhcpcd[769]: eth0: waiting for carrier991gitlab # [ 8.009042] dhcpcd[769]: eth0: carrier acquired992gitlab # [ 8.010143] redis-server[789]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.993gitlab # [ 8.012588] redis-server[789]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo994gitlab # [ 8.013888] redis-server[789]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=789, just started995gitlab # [ 8.015461] systemd[1]: Started Redis Server - redis-gitlab.996gitlab # [ 8.016351] redis-server[789]: Configuration loaded997gitlab # [ 8.017252] redis[789]: monotonic clock: POSIX clock_gettime998gitlab # [ 8.018182] dhcpcd[769]: DUID 00:01:00:01:32:2c:04:b4:52:54:00:12:34:56999gitlab # [ 8.019175] dhcpcd[769]: eth0: IAID 00:12:34:561000gitlab # [ 8.019964] dhcpcd[769]: eth0: adding address fe80::5054:ff:fe12:34561001gitlab # [ 8.020913] redis[789]: Running mode=standalone, port=0.1002gitlab # [ 8.022188] redis[789]: Server initialized1003gitlab # [ 8.023099] redis[789]: Ready to accept connections unix1004gitlab # [ 8.024111] redis[789]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1005gitlab # [ 8.027151] nginx-pre-start[793]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok1006gitlab # [ 8.028943] nginx-pre-start[793]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful1007gitlab # [ 8.032025] systemd[1]: Started Nginx Web Server.1008gitlab # [ 8.046313] postgresql-pre-start[795]: The files belonging to this database system will be owned by user "postgres".1009gitlab # [ 8.048373] postgresql-pre-start[795]: This user must also own the server process.1010gitlab # [ 8.050476] postgresql-pre-start[795]: The database cluster will be initialized with locale "en_US.UTF-8".1011gitlab # [ 8.051284] postgresql-pre-start[795]: The default database encoding has accordingly been set to "UTF8".1012gitlab # [ 8.051784] postgresql-pre-start[795]: The default text search configuration will be set to "english".1013gitlab # [ 8.052420] postgresql-pre-start[795]: Data page checksums are enabled.1014gitlab # [ 8.052956] postgresql-pre-start[795]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1015gitlab # [ 8.053711] postgresql-pre-start[795]: creating subdirectories ... ok1016gitlab # [ 8.054170] postgresql-pre-start[795]: selecting dynamic shared memory implementation ... posix1017gitlab # [ 8.093981] postgresql-pre-start[795]: selecting default "max_connections" ... 1001018gitlab # [ 8.128714] postgresql-pre-start[795]: selecting default "shared_buffers" ... 128MB1019gitlab # [ 8.406679] systemd[1]: Started gitlab-workhorse.service.1020gitlab # [ 8.521391] postgresql-pre-start[795]: selecting default time zone ... UTC1021gitlab # [ 8.523178] postgresql-pre-start[795]: creating configuration files ... ok1022gitlab # [ 8.614570] gitlab-workhorse[826]: time="2026-09-03T10:13:08Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41023gitlab # [ 8.615804] gitlab-workhorse[826]: time="2026-09-03T10:13:08Z" level=info msg="Using redis/go-redis"1024gitlab # [ 8.616967] gitlab-workhorse[826]: time="2026-09-03T10:13:08Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1025gitlab # [ 8.681921] postgresql-pre-start[795]: running bootstrap script ... ok1026gitlab # [ 9.080949] postgresql-pre-start[795]: performing post-bootstrap initialization ... ok1027gitlab # [ 9.189526] dhcpcd[769]: eth0: soliciting a DHCP lease1028gitlab # [ 9.349569] NET: Registered PF_PACKET protocol family1029gitlab # [ 9.212771] dhcpcd[769]: eth0: offered 10.0.2.15 from 10.0.2.21030gitlab # [ 9.220309] dhcpcd[769]: eth0: probing address 10.0.2.15/241031gitlab # [ 9.324237] postgresql-pre-start[795]: syncing data to disk ... ok1032gitlab # [ 9.326311] postgresql-pre-start[795]: initdb: warning: enabling "trust" authentication for local connections1033gitlab # [ 9.328101] postgresql-pre-start[795]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1034gitlab # [ 9.329291] postgresql-pre-start[795]: Success. You can now start the database server using:1035gitlab # [ 9.330467] postgresql-pre-start[795]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1036gitlab # [ 9.445439] postgres[852]: [852] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1037gitlab # [ 9.449879] postgres[852]: [852] LOG: listening on IPv6 address "::1", port 54321038gitlab # [ 9.452765] postgres[852]: [852] LOG: listening on IPv4 address "127.0.0.1", port 54321039gitlab # [ 9.455906] postgres[852]: [852] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1040gitlab # [ 9.461833] postgres[861]: [861] LOG: database system was shut down at 2026-09-03 10:13:09 GMT1041gitlab # [ 9.467582] postgres[852]: [852] LOG: database system is ready to accept connections1042gitlab # [ 9.473438] systemd[1]: Started PostgreSQL Server.1043gitlab # [ 9.477669] systemd[1]: Starting PostgreSQL Setup Scripts...1044gitlab # [ 9.756104] postgresql-setup-start[872]: CREATE DATABASE1045gitlab # [ 9.795135] postgresql-setup-start[877]: CREATE ROLE1046gitlab # [ 9.813521] postgresql-setup-start[879]: ALTER DATABASE1047gitlab # [ 9.861926] postgresql-setup-start[897]: CREATE ROLE1048gitlab # [ 9.864271] systemd[1]: Finished PostgreSQL Setup Scripts.1049gitlab # [ 9.866424] systemd[1]: Reached target PostgreSQL.1050gitlab # [ 9.868298] systemd[1]: Starting gitlab-postgresql.service...1051gitlab # [ 9.971437] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[896]: mkdir -p /var/gitlab/state/home/.ssh: OK1052gitlab # [ 9.973628] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[896]: chmod 700 /var/gitlab/state/home/.ssh: OK1053gitlab # [ 10.038443] systemd[1]: Finished gitlab-config.service.1054gitlab # [ 10.041675] systemd[1]: Started gitaly.service.1055gitlab # [ 10.044187] systemd[1]: Starting gitlab-db-config.service...1056gitlab # [ 10.452759] gitlab-postgresql-start[906]: CREATE DATABASE1057gitlab # [ 10.485204] gitlab-postgresql-start[933]: CREATE EXTENSION1058gitlab # [ 10.564488] gitlab-postgresql-start[935]: CREATE EXTENSION1059gitlab # [ 10.568617] systemd[1]: Finished gitlab-postgresql.service.1060gitlab # [ 10.592159] gitaly[920]: time="2026-09-03T10:13:10.580Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9201061gitlab # [ 10.593270] gitaly[920]: time="2026-09-03T10:13:10.580Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9201062gitlab # [ 10.594219] gitaly[920]: time="2026-09-03T10:13:10.580Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9201063gitlab # [ 10.594878] gitaly[920]: time="2026-09-03T10:13:10.580Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9201064gitlab # [ 10.595692] gitaly[920]: time="2026-09-03T10:13:10.580Z" level=info msg="Starting Gitaly" pid=920 version=1065gitlab # [ 10.658363] dhcpcd[769]: eth0: soliciting an IPv6 router1066gitlab # [ 10.658722] dhcpcd[769]: eth0: Router Advertisement from fe80::21067gitlab # [ 10.659261] dhcpcd[769]: eth0: adding address fec0::5054:ff:fe12:3456/641068gitlab # [ 10.659775] dhcpcd[769]: eth0: adding route to fec0::/641069gitlab # [ 10.660662] dhcpcd[769]: eth0: adding default route via fe80::21070gitlab # [ 10.752704] gitaly[920]: time="2026-09-03T10:13:10.741Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=150 pid=9201071gitlab # [ 10.753536] gitaly[920]: time="2026-09-03T10:13:10.742Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9201072gitlab # [ 10.754138] gitaly[920]: time="2026-09-03T10:13:10.742Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9201073gitlab # [ 10.754697] gitaly[920]: time="2026-09-03T10:13:10.742Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-920/git-exec-3446661433.d/git pid=9201074gitlab # [ 10.760525] gitaly[920]: time="2026-09-03T10:13:10.749Z" level=info msg="finished detecting git version" duration_ms=6 pid=9201075gitlab # [ 10.761236] gitaly[920]: time="2026-09-03T10:13:10.749Z" level=info msg="using Git version" pid=920 version=2.53.g1076gitlab # [ 10.761645] gitaly[920]: time="2026-09-03T10:13:10.749Z" level=info msg="clearing disk cache object folder" pid=920 storage=default1077gitlab # [ 10.762172] gitaly[920]: time="2026-09-03T10:13:10.750Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache581664719 pid=920 storage=default1078gitlab # [ 10.762725] gitaly[920]: time="2026-09-03T10:13:10.750Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache581664719 pid=920 storage=default1079gitlab # [ 10.762957] gitaly[920]: time="2026-09-03T10:13:10.750Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9201080gitlab # [ 10.763951] gitaly[920]: time="2026-09-03T10:13:10.750Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9201081gitlab # [ 10.764698] gitaly[920]: time="2026-09-03T10:13:10.752Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache581664719 clear_duration_ms=0 pid=920 storage=default1082gitlab # [ 10.812739] gitaly[920]: time="2026-09-03T10:13:10.801Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9201083gitlab # [ 10.813481] gitaly[920]: time="2026-09-03T10:13:10.801Z" level=info msg="finished tempdir cleaner walk" pid=920 storage=default time_ms=01084gitlab # [ 10.813618] gitaly[920]: time="2026-09-03T10:13:10.801Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=920 schema=unix1085gitlab # [ 10.814266] gitaly[920]: time="2026-09-03T10:13:10.801Z" level=info msg="listening at address" address=/run/gitaly/gitaly-920/sock.d/intern pid=920 schema=unix1086gitlab # [ 10.820537] gitaly[920]: time="2026-09-03T10:13:10.801Z" level=info msg="counting repositories" pid=9201087gitlab # [ 10.821234] gitaly[920]: time="2026-09-03T10:13:10.810Z" level=info msg="starting to count repositories" pid=920 storage_path=/var/gitlab/state/repositories1088gitlab # [ 10.821835] gitaly[920]: time="2026-09-03T10:13:10.810Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=920 storage_path=/var/gitlab/state/repositories1089gitlab # [ 10.822619] gitaly[920]: time="2026-09-03T10:13:10.810Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9201090gitlab # [ 10.828150] gitaly[920]: time="2026-09-03T10:13:10.817Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9201091gitlab # [ 10.829301] gitaly[920]: time="2026-09-03T10:13:10.817Z" level=info msg="Started Gitaly" duration_ms=236 pid=9201092gitlab # [ 10.830172] gitaly[920]: time="2026-09-03T10:13:10.817Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9201093gitlab # [ 10.830666] gitaly[920]: time="2026-09-03T10:13:10.818Z" level=info msg="maintenance: daily scheduled" pid=920 scheduled="2026-09-03 12:00:00 +0000 UTC"1094gitlab # [ 11.630517] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[923]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1095gitlab # [ 11.634884] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[923]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1096gitlab # [ 11.673979] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[923]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1097gitlab # [ 12.721459] gitaly[920]: time="2026-09-03T10:13:12.709Z" level=info msg="License database preloaded" duration_ms=2128 pid=9201098gitlab # [ 14.740125] dhcpcd[769]: eth0: leased 10.0.2.15 for 86400 seconds1099gitlab # [ 14.742283] dhcpcd[769]: eth0: adding route to 10.0.2.0/241100gitlab # [ 14.744227] dhcpcd[769]: eth0: adding default route via 10.0.2.21101gitlab # [ 14.857444] systemd[1]: Started DHCP Client.1102gitlab # [ 14.860179] systemd[1]: Reached target Network is Online.1103gitlab # [ 40.739222] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[923]: W, [2026-09-03T10:13:40.727391 #923] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1104gitlab # [ 42.654202] postgres[1308]: [1308] ERROR: relation "feature_gates" does not exist at character 5231105gitlab # [ 42.656328] postgres[1308]: [1308] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1106gitlab # [ 42.657495] postgres[1308]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1107gitlab # [ 42.659313] postgres[1308]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1108gitlab # [ 42.661343] postgres[1308]: attidentity AS identity,1109gitlab # [ 42.663196] postgres[1308]: attgenerated as attgenerated1110gitlab # [ 42.665070] postgres[1308]: FROM pg_attribute a1111gitlab # [ 42.666535] postgres[1308]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1112gitlab # [ 42.668404] postgres[1308]: LEFT JOIN pg_type t ON a.atttypid = t.oid1113gitlab # [ 42.670221] postgres[1308]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1114gitlab # [ 42.672070] postgres[1308]: WHERE a.attrelid = '"feature_gates"'::regclass1115gitlab # [ 42.673875] postgres[1308]: AND a.attnum > 0 AND NOT a.attisdropped1116gitlab # [ 42.676132] postgres[1308]: ORDER BY a.attnum1117gitlab # [ 42.677927] postgres[1308]:1118gitlab # [ 45.899420] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1341]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1119gitlab # [ 45.900697] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1341]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1120gitlab # [ 48.661921] gitlab-workhorse[826]: time="2026-09-03T10:13:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1121gitlab # [ 58.662837] gitlab-workhorse[826]: time="2026-09-03T10:13:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 61.890961] postgres[1407]: [1407] ERROR: relation "table_without_models" does not exist1123gitlab # [ 61.892557] postgres[1407]: [1407] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1124gitlab # [ 68.664124] gitlab-workhorse[826]: time="2026-09-03T10:14:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1125gitlab # [ 73.961182] systemd[1]: Finished gitlab-db-config.service.1126gitlab # [ 73.963436] systemd[1]: Started gitlab-sidekiq.service.1127gitlab # [ 73.965211] systemd[1]: Starting gitlab.service...1128gitlab # [ 74.311702] bundle[1592]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1129gitlab # [ 74.313336] bundle[1592]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1130gitlab # [ 74.315159] bundle[1592]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1131gitlab # [ 77.014318] sidekiq-cluster[1591]: {"severity":"INFO","time":"2026-09-03T10:14:16.986Z","message":"Starting cluster with 1 processes"}1132gitlab # [ 77.062582] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.032Z","pid":1592,"message":"* Enabling systemd notification integration"}1133gitlab # [ 77.063981] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.050Z","pid":1592,"message":"Puma starting in cluster mode..."}1134gitlab # [ 77.065181] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.050Z","pid":1592,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1135gitlab # [ 77.066461] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.050Z","pid":1592,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1136gitlab # [ 77.067391] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.050Z","pid":1592,"message":"* Min threads: 0"}1137gitlab # [ 77.068252] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.051Z","pid":1592,"message":"* Max threads: 4"}1138gitlab # [ 77.069447] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.051Z","pid":1592,"message":"* Environment: production"}1139gitlab # [ 77.070985] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.051Z","pid":1592,"message":"* Master PID: 1592"}1140gitlab # [ 77.073107] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.051Z","pid":1592,"message":"* Workers: 2"}1141gitlab # [ 77.074340] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.051Z","pid":1592,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1142gitlab # [ 77.075984] bundle[1592]: {"timestamp":"2026-09-03T10:14:17.051Z","pid":1592,"message":"* Preloading application"}1143gitlab # [ 78.673604] gitlab-workhorse[826]: time="2026-09-03T10:14:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1144gitlab # [ 88.674623] gitlab-workhorse[826]: time="2026-09-03T10:14:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 98.684953] gitlab-workhorse[826]: time="2026-09-03T10:14:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 99.140709] bundle[1592]: W, [2026-09-03T10:14:39.129275 #1592] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1147gitlab # [ 101.909605] sidekiq-cluster[1622]: 2026-09-03T10:14:41.898Z pid=1622 tid=wla INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1622, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433541120}1148gitlab # [ 102.082855] sidekiq-cluster[1622]: W, [2026-09-03T10:14:42.071627 #1622] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1149gitlab # [ 102.122302] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:42.111Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1150gitlab # [ 103.934757] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.922Z","message":"GitLab reliable fetch activated!"}1151gitlab # [ 103.935746] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.923Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1152gitlab # [ 103.960306] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.949Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1153gitlab # [ 103.964786] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.953Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1154gitlab # [ 103.969368] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.958Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1155gitlab # [ 103.973759] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.962Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1156gitlab # [ 103.978492] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.967Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1157gitlab # [ 103.982933] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.971Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1158gitlab # [ 103.987106] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.976Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1159gitlab # [ 103.990979] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.979Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1160gitlab # [ 103.994844] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.983Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1161gitlab # [ 103.998851] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.987Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1162gitlab # [ 104.002972] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.991Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1163gitlab # [ 104.007939] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:43.996Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1164gitlab # [ 104.011742] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.000Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1165gitlab # [ 104.016232] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.005Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1166gitlab # [ 104.021270] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.010Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1167gitlab # [ 104.026276] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.015Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1168gitlab # [ 104.032174] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.021Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1169gitlab # [ 104.037570] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.026Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1170gitlab # [ 104.042935] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.031Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1171gitlab # [ 104.049112] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.037Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1172gitlab # [ 104.055333] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.044Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1173gitlab # [ 104.060919] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.049Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1174gitlab # [ 104.069875] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.058Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1175gitlab # [ 104.075700] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.064Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1176gitlab # [ 104.081670] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.070Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1177gitlab # [ 104.087804] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.076Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1178gitlab # [ 104.093640] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.082Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1179gitlab # [ 104.099171] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.087Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1180gitlab # [ 104.104579] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.093Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1181gitlab # [ 104.110247] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.099Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1182gitlab # [ 104.114880] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.103Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1183gitlab # [ 104.120151] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.109Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1184gitlab # [ 104.124883] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.113Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1185gitlab # [ 104.130105] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.119Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1186gitlab # [ 104.135097] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.124Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1187gitlab # [ 104.139516] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.128Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1188gitlab # [ 104.144479] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.133Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1189gitlab # [ 104.149289] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.138Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1190gitlab # [ 104.154154] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.143Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1191gitlab # [ 104.158868] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.147Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1192gitlab # [ 104.163914] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.152Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1193gitlab # [ 104.169685] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.158Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1194gitlab # [ 104.175177] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.164Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1195gitlab # [ 104.180978] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.169Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1196gitlab # [ 104.187302] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.176Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1197gitlab # [ 104.192884] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.181Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1198gitlab # [ 104.197877] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.186Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1199gitlab # [ 104.204129] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.192Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1200gitlab # [ 104.209413] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.198Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1201gitlab # [ 104.214650] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.203Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1202gitlab # [ 104.219717] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.208Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1203gitlab # [ 104.224651] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.213Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1204gitlab # [ 104.239100] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.227Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1205gitlab # [ 104.241753] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.230Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1206gitlab # [ 104.245916] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.234Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1207gitlab # [ 104.248953] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.237Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1208gitlab # [ 104.251905] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.240Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1209gitlab # [ 104.255255] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.244Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1210gitlab # [ 104.258108] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.247Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1211gitlab # [ 104.261113] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.250Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1212gitlab # [ 104.264688] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.253Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1213gitlab # [ 104.267507] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.256Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1214gitlab # [ 104.271271] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.260Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1215gitlab # [ 104.274770] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.263Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1216gitlab # [ 104.277843] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.266Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1217gitlab # [ 104.281283] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.270Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1218gitlab # [ 104.284293] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.273Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1219gitlab # [ 104.287118] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.276Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1220gitlab # [ 104.289399] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.277Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1221gitlab # [ 104.294132] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.283Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1222gitlab # [ 104.298928] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.287Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1223gitlab # [ 104.303710] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.292Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1224gitlab # [ 104.333540] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.322Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1225gitlab # [ 104.337216] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.326Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1226gitlab # [ 104.339099] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.328Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1227gitlab # [ 104.341095] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.330Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1228gitlab # [ 104.343679] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.331Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1229gitlab # [ 104.345973] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.334Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1230gitlab # [ 104.348670] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.337Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1231gitlab # [ 104.351275] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.340Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1232gitlab # [ 104.353025] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.342Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1233gitlab # [ 104.355427] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.344Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1234gitlab # [ 104.357721] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.346Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1235gitlab # [ 104.360158] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.349Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1236gitlab # [ 104.362283] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.351Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1237gitlab # [ 104.364644] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.353Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1238gitlab # [ 104.367019] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.356Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1239gitlab # [ 104.369293] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.358Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1240gitlab # [ 104.371115] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.360Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1241gitlab # [ 104.372902] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.361Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1242gitlab # [ 104.374730] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.363Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1243gitlab # [ 104.376693] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.365Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1244gitlab # [ 104.379142] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.368Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1245gitlab # [ 104.381338] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.370Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1246gitlab # [ 104.384300] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.373Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1247gitlab # [ 104.386648] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.375Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1248gitlab # [ 104.389131] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.378Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1249gitlab # [ 104.391068] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.380Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1250gitlab # [ 104.400309] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.389Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1251gitlab # [ 104.402224] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.391Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1252gitlab # [ 104.404197] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.393Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1253gitlab # [ 104.406823] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.395Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1254gitlab # [ 104.409563] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.398Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1255gitlab # [ 104.411808] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.400Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1256gitlab # [ 104.414556] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.403Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1257gitlab # [ 104.416320] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.405Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1258gitlab # [ 104.418368] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.407Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1259gitlab # [ 104.420751] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.409Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1260gitlab # [ 104.423088] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:14:44.412Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1261gitlab # [ 107.911332] sidekiq-cluster[1622]: InitializerConnections Query: SET client_min_messages TO 'warning'1262gitlab # [ 107.912873] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1263gitlab # [ 107.913726] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1264gitlab # [ 107.914616] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1265gitlab # [ 107.915542] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1266gitlab # [ 107.916498] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1267gitlab # [ 107.917237] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1268gitlab # [ 107.925518] sidekiq-cluster[1622]: InitializerConnections Query: SET standard_conforming_strings = on1269gitlab # [ 107.926581] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1270gitlab # [ 107.927224] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1271gitlab # [ 107.927864] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1272gitlab # [ 107.928496] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1273gitlab # [ 107.929149] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1274gitlab # [ 107.929784] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1275gitlab # [ 107.936580] sidekiq-cluster[1622]: InitializerConnections Query: SET intervalstyle = iso_86011276gitlab # [ 107.937634] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1277gitlab # [ 107.938242] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1278gitlab # [ 107.938898] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1279gitlab # [ 107.939716] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1280gitlab # [ 107.940564] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1281gitlab # [ 107.941206] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1282gitlab # [ 107.955435] sidekiq-cluster[1622]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname1283gitlab # [ 107.956574] sidekiq-cluster[1622]: FROM pg_type as t1284gitlab # [ 107.958124] sidekiq-cluster[1622]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')1285gitlab # [ 107.959073] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1286gitlab # [ 107.959829] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1287gitlab # [ 107.960674] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1288gitlab # [ 107.961543] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1289gitlab # [ 107.962543] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1290gitlab # [ 107.963337] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1291gitlab # [ 107.973856] sidekiq-cluster[1622]: InitializerConnections Query: SET SESSION timezone TO 'UTC'1292gitlab # [ 107.974650] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1293gitlab # [ 107.975450] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1294gitlab # [ 107.976712] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1295gitlab # [ 107.980277] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1296gitlab # [ 107.981147] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1297gitlab # [ 107.981750] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1298gitlab # [ 107.990800] sidekiq-cluster[1622]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics'1299gitlab # [ 107.991534] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1300gitlab # [ 107.991956] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1301gitlab # [ 107.992581] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1302gitlab # [ 107.993047] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1303gitlab # [ 107.993531] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1304gitlab # [ 107.994052] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1305gitlab # [ 108.694666] gitlab-workhorse[826]: time="2026-09-03T10:14:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1306gitlab # [ 118.703586] gitlab-workhorse[826]: time="2026-09-03T10:14:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1307gitlab # [ 127.383596] postgres[2051]: [2051] ERROR: relation "table_without_models" does not exist1308gitlab # [ 127.385587] postgres[2051]: [2051] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1309gitlab # [ 128.705173] gitlab-workhorse[826]: time="2026-09-03T10:15:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1310gitlab # [ 128.713690] postgres[2062]: [2062] ERROR: relation "table_without_models" does not exist1311gitlab # [ 128.715142] postgres[2062]: [2062] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1312gitlab # [ 129.909483] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:09.897Z","message":"Booted Rails 7.2.3.1 application in production environment"}1313gitlab # [ 129.913367] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:09.902Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1314gitlab # [ 129.913968] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:09.902Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1315gitlab # [ 129.914602] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:09.902Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1316gitlab # [ 129.920137] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:09.908Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1317gitlab # [ 131.160495] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:11.148Z","message":"Cleaning working queues"}1318gitlab # [ 131.161804] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:11.150Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e0218eb21e38963ac3852f76","created_at":"2026-09-03T10:14:13.523Z","trace_propagation_headers":{"sentry-trace":"9623a0283d6740039e1a21337a6c247c-826642d6a1134905","baggage":"sentry-trace_id=9623a0283d6740039e1a21337a6c247c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"1338d0f901fe523837c516999af34cd5","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-03T10:14:13.531Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e0218eb21e38963ac3852f76: start","job_status":"start","queue_duration_s":57.618513,"scheduling_latency_s":57.618552}1319gitlab # [ 131.332632] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:11.321Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e0218eb21e38963ac3852f76","created_at":"2026-09-03T10:14:13.523Z","trace_propagation_headers":{"sentry-trace":"9623a0283d6740039e1a21337a6c247c-826642d6a1134905","baggage":"sentry-trace_id=9623a0283d6740039e1a21337a6c247c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"1338d0f901fe523837c516999af34cd5","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-03T10:14:13.531Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e0218eb21e38963ac3852f76: done: 0.171188 sec","job_status":"done","queue_duration_s":57.618513,"scheduling_latency_s":57.618552,"redis_calls":31,"redis_duration_s":0.006893,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.004684,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001199,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.085894437,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.171188,"completed_at":"2026-09-03T10:15:11.320Z","load_balancing_strategy":"primary","db_duration_s":0.061111,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1320gitlab # [ 131.565250] bundle[1592]: {"timestamp":"2026-09-03T10:15:11.553Z","pid":1592,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1321gitlab # [ 131.567298] bundle[1592]: {"timestamp":"2026-09-03T10:15:11.553Z","pid":1592,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1322gitlab # [ 131.568588] bundle[1592]: {"timestamp":"2026-09-03T10:15:11.553Z","pid":1592,"message":"! #\u003cThread:0x000076cf796d7fa8 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1323gitlab # [ 131.570006] bundle[1592]: {"timestamp":"2026-09-03T10:15:11.553Z","pid":1592,"message":"! #\u003cThread:0x000076cf7991f568 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1324gitlab # [ 131.571223] bundle[1592]: {"timestamp":"2026-09-03T10:15:11.553Z","pid":1592,"message":"Use Ctrl-C to stop"}1325gitlab # [ 131.572518] bundle[1592]: {"timestamp":"2026-09-03T10:15:11.554Z","pid":1592,"message":"Sending status to systemd every 1.0 sec"}1326gitlab # [ 131.892619] bundle[1592]: {"timestamp":"2026-09-03T10:15:11.881Z","pid":1592,"message":"- Worker 1 (PID: 2167) booted in 0.25s, phase: 0"}1327gitlab # [ 131.904861] bundle[1592]: {"timestamp":"2026-09-03T10:15:11.893Z","pid":1592,"message":"- Worker 0 (PID: 2163) booted in 0.28s, phase: 0"}1328gitlab # [ 131.913643] systemd[1]: Started gitlab.service.1329gitlab # [ 131.915250] systemd[1]: Reached target Common target for all GitLab services..1330gitlab # [ 131.917775] systemd[1]: Starting setup-gitlab.service...1331gitlab # [ 131.941351] setup-gitlab-start[2191]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1332gitlab # [ 134.191878] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:15:14 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 22221333gitlab # [ 134.198535] setup-gitlab-start[2191]: + TOKEN=nixbot-test-token-of-20-chars1334gitlab # [ 134.200217] setup-gitlab-start[2191]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1335gitlab # [ 134.225675] runuser[2209]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1336gitlab # [ 134.598877] setup-gitlab-start[2210]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1337gitlab # [ 134.603768] setup-gitlab-start[2210]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1338gitlab # [ 134.608550] setup-gitlab-start[2210]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1339gitlab # [ 135.165318] setup-gitlab-start[2210]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1340gitlab # [ 135.170291] setup-gitlab-start[2210]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1341gitlab # [ 135.174982] setup-gitlab-start[2210]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1342gitlab # [ 143.557824] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:23.545Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"39b3bbd9d2e5696a31a8be30","created_at":"2026-09-03T10:15:23.516Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.543Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-39b3bbd9d2e5696a31a8be30: start","job_status":"start","queue_duration_s":0.001778,"scheduling_latency_s":0.001811}1343gitlab # [ 143.746392] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:23.734Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"39b3bbd9d2e5696a31a8be30","created_at":"2026-09-03T10:15:23.516Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.543Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-39b3bbd9d2e5696a31a8be30: done: 0.189497 sec","job_status":"done","queue_duration_s":0.001778,"scheduling_latency_s":0.001811,"redis_calls":71,"redis_duration_s":0.084579,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002552,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001078,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.07939,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001559,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000417371,"duration_s":0.189497,"completed_at":"2026-09-03T10:15:23.734Z","load_balancing_strategy":"primary","db_duration_s":0.002974,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 143.786962] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:23.775Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"07b776c278febcfb90838535","created_at":"2026-09-03T10:15:23.552Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d14e456c6b5fa10acfa01ec171a79337","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.564Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-07b776c278febcfb90838535: start","job_status":"start","queue_duration_s":0.210764,"scheduling_latency_s":0.210785}1345gitlab # [ 143.904721] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:23.892Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"07b776c278febcfb90838535","created_at":"2026-09-03T10:15:23.552Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d14e456c6b5fa10acfa01ec171a79337","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.564Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-07b776c278febcfb90838535: done: 0.117186 sec","job_status":"done","queue_duration_s":0.210764,"scheduling_latency_s":0.210785,"redis_calls":14,"redis_duration_s":0.013351,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00542,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00505,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002881,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001224178,"exclusive_lock_hold_duration_s":0.061113912,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.117186,"completed_at":"2026-09-03T10:15:23.892Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 143.968644] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:23.957Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f533b3802918f77bab2d9d65","created_at":"2026-09-03T10:15:23.567Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"725a2a52a24932c7183e5a394839be5c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f533b3802918f77bab2d9d65: start","job_status":"start","queue_duration_s":0.364704,"scheduling_latency_s":0.364746}1347gitlab # [ 144.078710] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.067Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"f533b3802918f77bab2d9d65","created_at":"2026-09-03T10:15:23.567Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"725a2a52a24932c7183e5a394839be5c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-f533b3802918f77bab2d9d65: done: 0.110145 sec","job_status":"done","queue_duration_s":0.364704,"scheduling_latency_s":0.364746,"redis_calls":13,"redis_duration_s":0.013293000000000001,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001177,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007879,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004237,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.019898060000000002,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.110145,"completed_at":"2026-09-03T10:15:24.067Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015022,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1348gitlab # [ 144.144218] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"8b9e439f7b441b7c88b3f506","created_at":"2026-09-03T10:15:23.597Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b8b9f3b1fc80fe6115d821e73f37155","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.605Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-8b9e439f7b441b7c88b3f506: start","job_status":"start","queue_duration_s":0.468835,"scheduling_latency_s":0.468876}1349gitlab # [ 144.186605] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.174Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"8b9e439f7b441b7c88b3f506","created_at":"2026-09-03T10:15:23.597Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b8b9f3b1fc80fe6115d821e73f37155","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.605Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-8b9e439f7b441b7c88b3f506: done: 0.100792 sec","job_status":"done","queue_duration_s":0.468835,"scheduling_latency_s":0.468876,"redis_calls":17,"redis_duration_s":0.015576,"redis_read_bytes":15,"redis_write_bytes":2007,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.007336,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1150,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005361,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.002065,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000570184,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.100792,"completed_at":"2026-09-03T10:15:24.174Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 144.227427] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.177Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2979b19bc8bf962ccdd42713","created_at":"2026-09-03T10:15:23.611Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2514c37e307651472c0ddc27747c9214","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.618Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2979b19bc8bf962ccdd42713: start","job_status":"start","queue_duration_s":0.559401,"scheduling_latency_s":0.559437}1351gitlab # [ 144.228035] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.200Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1352gitlab # [ 144.228506] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2979b19bc8bf962ccdd42713","created_at":"2026-09-03T10:15:23.611Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2514c37e307651472c0ddc27747c9214","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.618Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2979b19bc8bf962ccdd42713: done: 0.033426 sec","job_status":"done","queue_duration_s":0.559401,"scheduling_latency_s":0.559437,"redis_calls":10,"redis_duration_s":0.004089,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002561,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000424356,"duration_s":0.033426,"completed_at":"2026-09-03T10:15:24.211Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001982,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 144.242060] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.212Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f0c39aa4a25b32606ac5db3e","created_at":"2026-09-03T10:15:23.628Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34af892ed6d8c0be3055ec95f8ad3214","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.633Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f0c39aa4a25b32606ac5db3e: start","job_status":"start","queue_duration_s":0.579539,"scheduling_latency_s":0.579554}1354gitlab # [ 144.329329] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.317Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f0c39aa4a25b32606ac5db3e","created_at":"2026-09-03T10:15:23.628Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34af892ed6d8c0be3055ec95f8ad3214","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.633Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f0c39aa4a25b32606ac5db3e: done: 0.104532 sec","job_status":"done","queue_duration_s":0.579539,"scheduling_latency_s":0.579554,"redis_calls":10,"redis_duration_s":0.006657,"redis_read_bytes":9,"redis_write_bytes":1109,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000598,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003991,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000552026,"duration_s":0.104532,"completed_at":"2026-09-03T10:15:24.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.063852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 144.351107] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"828c86847d395b893678ba13","created_at":"2026-09-03T10:15:23.640Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d9052be0f10f79c135a68a508e13248","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:15:23.646Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-828c86847d395b893678ba13: start","job_status":"start","queue_duration_s":0.673547,"scheduling_latency_s":0.673573}1356gitlab # [ 144.351852] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.334Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1357gitlab # [ 144.354361] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"828c86847d395b893678ba13","created_at":"2026-09-03T10:15:23.640Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d9052be0f10f79c135a68a508e13248","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:15:23.646Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-828c86847d395b893678ba13: done: 0.023237 sec","job_status":"done","queue_duration_s":0.673547,"scheduling_latency_s":0.673573,"redis_calls":10,"redis_duration_s":0.0029249999999999996,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000551,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001594,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00078,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000455365,"duration_s":0.023237,"completed_at":"2026-09-03T10:15:24.343Z","load_balancing_strategy":"primary","db_duration_s":0.00055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 144.406601] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.344Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"12c454dbc6e4e2549eeba040","created_at":"2026-09-03T10:15:23.651Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"301c618b0ea86b8024c296854123ac20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.656Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-12c454dbc6e4e2549eeba040: start","job_status":"start","queue_duration_s":0.687163,"scheduling_latency_s":0.687176}1359gitlab # [ 144.407620] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.351Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1360gitlab # [ 144.408531] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"12c454dbc6e4e2549eeba040","created_at":"2026-09-03T10:15:23.651Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"301c618b0ea86b8024c296854123ac20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.656Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-12c454dbc6e4e2549eeba040: done: 0.012625 sec","job_status":"done","queue_duration_s":0.687163,"scheduling_latency_s":0.687176,"redis_calls":10,"redis_duration_s":0.0013,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006927,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000203378,"duration_s":0.012625,"completed_at":"2026-09-03T10:15:24.356Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 144.484150] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.357Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7bf6116b849b14e1222da540","created_at":"2026-09-03T10:15:23.687Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7e710c0a35f6f1f970837464b5e4f61","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.695Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7bf6116b849b14e1222da540: start","job_status":"start","queue_duration_s":0.662408,"scheduling_latency_s":0.662421}1362gitlab # [ 144.484944] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7bf6116b849b14e1222da540","created_at":"2026-09-03T10:15:23.687Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7e710c0a35f6f1f970837464b5e4f61","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.695Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7bf6116b849b14e1222da540: done: 0.03176 sec","job_status":"done","queue_duration_s":0.662408,"scheduling_latency_s":0.662421,"redis_calls":12,"redis_duration_s":0.001695,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00066,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000362616,"exclusive_lock_hold_duration_s":0.014130567,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.03176,"completed_at":"2026-09-03T10:15:24.389Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014811,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 144.546305] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"d7a4915257af80dfbabe422b","created_at":"2026-09-03T10:15:23.706Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba27cd4125507464a08fdeff0431e93d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.713Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-d7a4915257af80dfbabe422b: start","job_status":"start","queue_duration_s":0.677389,"scheduling_latency_s":0.677402}1364gitlab # [ 144.547301] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"d7a4915257af80dfbabe422b","created_at":"2026-09-03T10:15:23.706Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba27cd4125507464a08fdeff0431e93d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.713Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-d7a4915257af80dfbabe422b: done: 0.016358 sec","job_status":"done","queue_duration_s":0.677389,"scheduling_latency_s":0.677402,"redis_calls":11,"redis_duration_s":0.0014,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000733,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000403,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000225448,"duration_s":0.016358,"completed_at":"2026-09-03T10:15:24.406Z","load_balancing_strategy":"primary","db_duration_s":0.001795,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 144.560328] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c416013872646a27d80615f7","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c416013872646a27d80615f7: start","job_status":"start","queue_duration_s":0.683268,"scheduling_latency_s":0.683281}1366gitlab # [ 144.598024] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c416013872646a27d80615f7","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c416013872646a27d80615f7: done: 0.014372 sec","job_status":"done","queue_duration_s":0.683268,"scheduling_latency_s":0.683281,"redis_calls":16,"redis_duration_s":0.002185,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000288,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000623,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001274,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200305,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014372,"completed_at":"2026-09-03T10:15:24.421Z","load_balancing_strategy":"primary","db_duration_s":0.00026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 144.637946] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7208bd912bc6a8aca907177","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7208bd912bc6a8aca907177: start","job_status":"start","queue_duration_s":0.698629,"scheduling_latency_s":0.698642}1368gitlab # [ 144.638680] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7208bd912bc6a8aca907177","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7208bd912bc6a8aca907177: done: 0.010035 sec","job_status":"done","queue_duration_s":0.698629,"scheduling_latency_s":0.698642,"redis_calls":13,"redis_duration_s":0.0016020000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010035,"completed_at":"2026-09-03T10:15:24.433Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 144.649272] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a3b25ea5c9a440eecdfd3ff","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a3b25ea5c9a440eecdfd3ff: start","job_status":"start","queue_duration_s":0.70967,"scheduling_latency_s":0.709683}1370gitlab # [ 144.659850] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a3b25ea5c9a440eecdfd3ff","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a3b25ea5c9a440eecdfd3ff: done: 0.010173 sec","job_status":"done","queue_duration_s":0.70967,"scheduling_latency_s":0.709683,"redis_calls":13,"redis_duration_s":0.001719,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010173,"completed_at":"2026-09-03T10:15:24.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 144.707629] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a568ad37dd52d6ccbfa92e09","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a568ad37dd52d6ccbfa92e09: start","job_status":"start","queue_duration_s":0.720815,"scheduling_latency_s":0.720828}1372gitlab # [ 144.708480] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a568ad37dd52d6ccbfa92e09","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a568ad37dd52d6ccbfa92e09: done: 0.009601 sec","job_status":"done","queue_duration_s":0.720815,"scheduling_latency_s":0.720828,"redis_calls":13,"redis_duration_s":0.001548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009601,"completed_at":"2026-09-03T10:15:24.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 144.747665] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0e2f743362e51e2db6a4df0","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0e2f743362e51e2db6a4df0: start","job_status":"start","queue_duration_s":0.731261,"scheduling_latency_s":0.731274}1374gitlab # [ 144.748593] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0e2f743362e51e2db6a4df0","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0e2f743362e51e2db6a4df0: done: 0.010091 sec","job_status":"done","queue_duration_s":0.731261,"scheduling_latency_s":0.731274,"redis_calls":13,"redis_duration_s":0.0015509999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010091,"completed_at":"2026-09-03T10:15:24.465Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 144.786709] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0e1a48a6d214b057c90a4e3","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0e1a48a6d214b057c90a4e3: start","job_status":"start","queue_duration_s":0.742196,"scheduling_latency_s":0.742209}1376gitlab # [ 144.792674] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0e1a48a6d214b057c90a4e3","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0e1a48a6d214b057c90a4e3: done: 0.010146 sec","job_status":"done","queue_duration_s":0.742196,"scheduling_latency_s":0.742209,"redis_calls":13,"redis_duration_s":0.00169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010146,"completed_at":"2026-09-03T10:15:24.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 144.834304] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d64a8657711c6ad5a159b100","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d64a8657711c6ad5a159b100: start","job_status":"start","queue_duration_s":0.753233,"scheduling_latency_s":0.753246}1378gitlab # [ 144.836727] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d64a8657711c6ad5a159b100","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d64a8657711c6ad5a159b100: done: 0.009826 sec","job_status":"done","queue_duration_s":0.753233,"scheduling_latency_s":0.753246,"redis_calls":13,"redis_duration_s":0.0015700000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009826,"completed_at":"2026-09-03T10:15:24.487Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 144.888178] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74fb12f76a05391842ce72b8","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74fb12f76a05391842ce72b8: start","job_status":"start","queue_duration_s":0.763899,"scheduling_latency_s":0.763913}1380gitlab # [ 144.888882] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.498Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74fb12f76a05391842ce72b8","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74fb12f76a05391842ce72b8: done: 0.009802 sec","job_status":"done","queue_duration_s":0.763899,"scheduling_latency_s":0.763913,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009802,"completed_at":"2026-09-03T10:15:24.498Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 144.927502] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2ee022262ff92aac48bac0e","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2ee022262ff92aac48bac0e: start","job_status":"start","queue_duration_s":0.774657,"scheduling_latency_s":0.77467}1382gitlab # [ 144.928248] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2ee022262ff92aac48bac0e","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2ee022262ff92aac48bac0e: done: 0.011616 sec","job_status":"done","queue_duration_s":0.774657,"scheduling_latency_s":0.77467,"redis_calls":13,"redis_duration_s":0.0018670000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011616,"completed_at":"2026-09-03T10:15:24.510Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 144.968140] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d79f5761ae8e0fe515cc64cd","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d79f5761ae8e0fe515cc64cd: start","job_status":"start","queue_duration_s":0.787482,"scheduling_latency_s":0.787497}1384gitlab # [ 144.971738] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d79f5761ae8e0fe515cc64cd","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d79f5761ae8e0fe515cc64cd: done: 0.012542 sec","job_status":"done","queue_duration_s":0.787482,"scheduling_latency_s":0.787497,"redis_calls":13,"redis_duration_s":0.002354,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001627,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012542,"completed_at":"2026-09-03T10:15:24.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 145.016870] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a509b1efcfb6cd6a661bf95","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a509b1efcfb6cd6a661bf95: start","job_status":"start","queue_duration_s":0.801456,"scheduling_latency_s":0.801471}1386gitlab # [ 145.017765] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a509b1efcfb6cd6a661bf95","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a509b1efcfb6cd6a661bf95: done: 0.013357 sec","job_status":"done","queue_duration_s":0.801456,"scheduling_latency_s":0.801471,"redis_calls":13,"redis_duration_s":0.002325,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001621,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013357,"completed_at":"2026-09-03T10:15:24.539Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 145.063910] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ede2b9c24f6e067675ba1d2a","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ede2b9c24f6e067675ba1d2a: start","job_status":"start","queue_duration_s":0.816379,"scheduling_latency_s":0.816395}1388gitlab # [ 145.065021] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ede2b9c24f6e067675ba1d2a","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ede2b9c24f6e067675ba1d2a: done: 0.013444 sec","job_status":"done","queue_duration_s":0.816379,"scheduling_latency_s":0.816395,"redis_calls":13,"redis_duration_s":0.0019190000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001248,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013444,"completed_at":"2026-09-03T10:15:24.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 145.131572] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02f0356bdae113bb29c6b5d4","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02f0356bdae113bb29c6b5d4: start","job_status":"start","queue_duration_s":0.831545,"scheduling_latency_s":0.831574}1390gitlab # [ 145.132461] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02f0356bdae113bb29c6b5d4","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02f0356bdae113bb29c6b5d4: done: 0.018674 sec","job_status":"done","queue_duration_s":0.831545,"scheduling_latency_s":0.831574,"redis_calls":13,"redis_duration_s":0.002285,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000675,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018674,"completed_at":"2026-09-03T10:15:24.574Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 145.173513] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edc5e808599c6fb9de4d6abe","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edc5e808599c6fb9de4d6abe: start","job_status":"start","queue_duration_s":0.8513,"scheduling_latency_s":0.851314}1392gitlab # [ 145.174160] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edc5e808599c6fb9de4d6abe","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edc5e808599c6fb9de4d6abe: done: 0.009198 sec","job_status":"done","queue_duration_s":0.8513,"scheduling_latency_s":0.851314,"redis_calls":13,"redis_duration_s":0.001345,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00043,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009198,"completed_at":"2026-09-03T10:15:24.584Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 145.219058] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e696b6d1e88e6a40c28286f0","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e696b6d1e88e6a40c28286f0: start","job_status":"start","queue_duration_s":0.861267,"scheduling_latency_s":0.86128}1394gitlab # [ 145.219804] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e696b6d1e88e6a40c28286f0","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e696b6d1e88e6a40c28286f0: done: 0.009511 sec","job_status":"done","queue_duration_s":0.861267,"scheduling_latency_s":0.86128,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009511,"completed_at":"2026-09-03T10:15:24.595Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 145.270956] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85a60310c2d728c974ac56e0","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85a60310c2d728c974ac56e0: start","job_status":"start","queue_duration_s":0.871541,"scheduling_latency_s":0.871553}1396gitlab # [ 145.271690] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85a60310c2d728c974ac56e0","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85a60310c2d728c974ac56e0: done: 0.009412 sec","job_status":"done","queue_duration_s":0.871541,"scheduling_latency_s":0.871553,"redis_calls":13,"redis_duration_s":0.001451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000442,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009412,"completed_at":"2026-09-03T10:15:24.605Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 145.281557] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.606Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02f3bb2f981655b7d138df99","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02f3bb2f981655b7d138df99: start","job_status":"start","queue_duration_s":0.881823,"scheduling_latency_s":0.881835}1398gitlab # [ 145.292672] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02f3bb2f981655b7d138df99","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02f3bb2f981655b7d138df99: done: 0.009236 sec","job_status":"done","queue_duration_s":0.881823,"scheduling_latency_s":0.881835,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009236,"completed_at":"2026-09-03T10:15:24.615Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 145.350819] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c24b6e5c1ff993f8b90aa7c","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c24b6e5c1ff993f8b90aa7c: start","job_status":"start","queue_duration_s":0.891814,"scheduling_latency_s":0.891826}1400gitlab # [ 145.351661] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c24b6e5c1ff993f8b90aa7c","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c24b6e5c1ff993f8b90aa7c: done: 0.009254 sec","job_status":"done","queue_duration_s":0.891814,"scheduling_latency_s":0.891826,"redis_calls":13,"redis_duration_s":0.001544,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009254,"completed_at":"2026-09-03T10:15:24.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 145.374126] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5786e0d9ae9c2a454cd70902","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5786e0d9ae9c2a454cd70902: start","job_status":"start","queue_duration_s":0.90196,"scheduling_latency_s":0.901973}1402gitlab # [ 145.385252] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5786e0d9ae9c2a454cd70902","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5786e0d9ae9c2a454cd70902: done: 0.009446 sec","job_status":"done","queue_duration_s":0.90196,"scheduling_latency_s":0.901973,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009446,"completed_at":"2026-09-03T10:15:24.635Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 145.454710] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9de438c10cc114d597ad0932","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9de438c10cc114d597ad0932: start","job_status":"start","queue_duration_s":0.912222,"scheduling_latency_s":0.912234}1404gitlab # [ 145.455574] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9de438c10cc114d597ad0932","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9de438c10cc114d597ad0932: done: 0.009626 sec","job_status":"done","queue_duration_s":0.912222,"scheduling_latency_s":0.912234,"redis_calls":13,"redis_duration_s":0.001753,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009626,"completed_at":"2026-09-03T10:15:24.646Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 145.496770] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85dc7efd4957c0a55d441ce9","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85dc7efd4957c0a55d441ce9: start","job_status":"start","queue_duration_s":0.922797,"scheduling_latency_s":0.92281}1406gitlab # [ 145.497206] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85dc7efd4957c0a55d441ce9","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85dc7efd4957c0a55d441ce9: done: 0.009396 sec","job_status":"done","queue_duration_s":0.922797,"scheduling_latency_s":0.92281,"redis_calls":13,"redis_duration_s":0.001548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009396,"completed_at":"2026-09-03T10:15:24.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 145.512577] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a3b48e757d7d97c428f0682","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a3b48e757d7d97c428f0682: start","job_status":"start","queue_duration_s":0.933274,"scheduling_latency_s":0.933288}1408gitlab # [ 145.550126] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a3b48e757d7d97c428f0682","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a3b48e757d7d97c428f0682: done: 0.009492 sec","job_status":"done","queue_duration_s":0.933274,"scheduling_latency_s":0.933288,"redis_calls":13,"redis_duration_s":0.0016150000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009492,"completed_at":"2026-09-03T10:15:24.667Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 145.590118] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2133386e37672b2c9beded35","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2133386e37672b2c9beded35: start","job_status":"start","queue_duration_s":0.943838,"scheduling_latency_s":0.94385}1410gitlab # [ 145.590887] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2133386e37672b2c9beded35","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2133386e37672b2c9beded35: done: 0.009224 sec","job_status":"done","queue_duration_s":0.943838,"scheduling_latency_s":0.94385,"redis_calls":13,"redis_duration_s":0.00142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009224,"completed_at":"2026-09-03T10:15:24.677Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 145.602674] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb17ab390fabebe1e6c6f69c","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb17ab390fabebe1e6c6f69c: start","job_status":"start","queue_duration_s":0.954012,"scheduling_latency_s":0.954025}1412gitlab # [ 145.614652] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb17ab390fabebe1e6c6f69c","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb17ab390fabebe1e6c6f69c: done: 0.009719 sec","job_status":"done","queue_duration_s":0.954012,"scheduling_latency_s":0.954025,"redis_calls":13,"redis_duration_s":0.0016049999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009719,"completed_at":"2026-09-03T10:15:24.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 145.649294] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e8d23f45f101cbfe0056b46","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e8d23f45f101cbfe0056b46: start","job_status":"start","queue_duration_s":0.965247,"scheduling_latency_s":0.965278}1414gitlab # [ 145.696814] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e8d23f45f101cbfe0056b46","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e8d23f45f101cbfe0056b46: done: 0.012414 sec","job_status":"done","queue_duration_s":0.965247,"scheduling_latency_s":0.965278,"redis_calls":13,"redis_duration_s":0.0017690000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012414,"completed_at":"2026-09-03T10:15:24.701Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 145.698283] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81a9114d71e51611b46ad3e1","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81a9114d71e51611b46ad3e1: start","job_status":"start","queue_duration_s":0.978474,"scheduling_latency_s":0.978486}1416gitlab # [ 145.739570] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81a9114d71e51611b46ad3e1","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81a9114d71e51611b46ad3e1: done: 0.009571 sec","job_status":"done","queue_duration_s":0.978474,"scheduling_latency_s":0.978486,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000476,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009571,"completed_at":"2026-09-03T10:15:24.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 145.741433] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9956e0ab132ba081e527337","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9956e0ab132ba081e527337: start","job_status":"start","queue_duration_s":0.988838,"scheduling_latency_s":0.988851}1418gitlab # [ 145.752737] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9956e0ab132ba081e527337","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9956e0ab132ba081e527337: done: 0.009457 sec","job_status":"done","queue_duration_s":0.988838,"scheduling_latency_s":0.988851,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009457,"completed_at":"2026-09-03T10:15:24.722Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 145.791751] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"769c9b0e6981f3333a44eb40","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-769c9b0e6981f3333a44eb40: start","job_status":"start","queue_duration_s":0.999105,"scheduling_latency_s":0.999117}1420gitlab # [ 145.798631] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"769c9b0e6981f3333a44eb40","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-769c9b0e6981f3333a44eb40: done: 0.009688 sec","job_status":"done","queue_duration_s":0.999105,"scheduling_latency_s":0.999117,"redis_calls":13,"redis_duration_s":0.0014969999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009688,"completed_at":"2026-09-03T10:15:24.733Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 145.851648] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cff43591730b6103ff103fff","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cff43591730b6103ff103fff: start","job_status":"start","queue_duration_s":1.009593,"scheduling_latency_s":1.009605}1422gitlab # [ 145.852605] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cff43591730b6103ff103fff","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cff43591730b6103ff103fff: done: 0.009905 sec","job_status":"done","queue_duration_s":1.009593,"scheduling_latency_s":1.009605,"redis_calls":13,"redis_duration_s":0.001731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001219,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009905,"completed_at":"2026-09-03T10:15:24.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 145.879830] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69d1395ea7dc534355ee5cb4","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69d1395ea7dc534355ee5cb4: start","job_status":"start","queue_duration_s":1.020322,"scheduling_latency_s":1.020335}1424gitlab # [ 145.891072] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69d1395ea7dc534355ee5cb4","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69d1395ea7dc534355ee5cb4: done: 0.009516 sec","job_status":"done","queue_duration_s":1.020322,"scheduling_latency_s":1.020335,"redis_calls":13,"redis_duration_s":0.0016460000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009516,"completed_at":"2026-09-03T10:15:24.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 145.940685] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67246aacbe57d46b8ef35535","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67246aacbe57d46b8ef35535: start","job_status":"start","queue_duration_s":1.030621,"scheduling_latency_s":1.030635}1426gitlab # [ 145.941625] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67246aacbe57d46b8ef35535","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67246aacbe57d46b8ef35535: done: 0.011054 sec","job_status":"done","queue_duration_s":1.030621,"scheduling_latency_s":1.030635,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011054,"completed_at":"2026-09-03T10:15:24.766Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 145.992433] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed18a813952c2e2a9f4811d3","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed18a813952c2e2a9f4811d3: start","job_status":"start","queue_duration_s":1.043088,"scheduling_latency_s":1.043107}1428gitlab # [ 145.993204] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed18a813952c2e2a9f4811d3","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed18a813952c2e2a9f4811d3: done: 0.010438 sec","job_status":"done","queue_duration_s":1.043088,"scheduling_latency_s":1.043107,"redis_calls":13,"redis_duration_s":0.001677,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010438,"completed_at":"2026-09-03T10:15:24.777Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 146.034294] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4524aac0008d2ec14455fd70","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4524aac0008d2ec14455fd70: start","job_status":"start","queue_duration_s":1.054385,"scheduling_latency_s":1.054399}1430gitlab # [ 146.034889] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4524aac0008d2ec14455fd70","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4524aac0008d2ec14455fd70: done: 0.009219 sec","job_status":"done","queue_duration_s":1.054385,"scheduling_latency_s":1.054399,"redis_calls":13,"redis_duration_s":0.001576,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009219,"completed_at":"2026-09-03T10:15:24.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 146.082275] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54c48e512ffcd91da4333f64","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54c48e512ffcd91da4333f64: start","job_status":"start","queue_duration_s":1.064515,"scheduling_latency_s":1.064529}1432gitlab # [ 146.082948] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54c48e512ffcd91da4333f64","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54c48e512ffcd91da4333f64: done: 0.009132 sec","job_status":"done","queue_duration_s":1.064515,"scheduling_latency_s":1.064529,"redis_calls":13,"redis_duration_s":0.00159,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009132,"completed_at":"2026-09-03T10:15:24.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 146.112210] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea9b373743aa97bea4b94666","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea9b373743aa97bea4b94666: start","job_status":"start","queue_duration_s":1.07454,"scheduling_latency_s":1.074553}1434gitlab # [ 146.123517] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea9b373743aa97bea4b94666","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea9b373743aa97bea4b94666: done: 0.009432 sec","job_status":"done","queue_duration_s":1.07454,"scheduling_latency_s":1.074553,"redis_calls":13,"redis_duration_s":0.001653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009432,"completed_at":"2026-09-03T10:15:24.808Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 146.175645] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12b43bab04b6e2440fbbcbf8","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12b43bab04b6e2440fbbcbf8: start","job_status":"start","queue_duration_s":1.084961,"scheduling_latency_s":1.084974}1436gitlab # [ 146.176282] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12b43bab04b6e2440fbbcbf8","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12b43bab04b6e2440fbbcbf8: done: 0.009621 sec","job_status":"done","queue_duration_s":1.084961,"scheduling_latency_s":1.084974,"redis_calls":13,"redis_duration_s":0.001735,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009621,"completed_at":"2026-09-03T10:15:24.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 146.225566] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8563910f64ed16151fe0da0f","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8563910f64ed16151fe0da0f: start","job_status":"start","queue_duration_s":1.095423,"scheduling_latency_s":1.095436}1438gitlab # [ 146.226490] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8563910f64ed16151fe0da0f","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8563910f64ed16151fe0da0f: done: 0.009734 sec","job_status":"done","queue_duration_s":1.095423,"scheduling_latency_s":1.095436,"redis_calls":13,"redis_duration_s":0.001624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009734,"completed_at":"2026-09-03T10:15:24.829Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 146.267193] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b96f30707ad8ea9208296979","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b96f30707ad8ea9208296979: start","job_status":"start","queue_duration_s":1.105989,"scheduling_latency_s":1.106002}1440gitlab # [ 146.268169] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b96f30707ad8ea9208296979","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b96f30707ad8ea9208296979: done: 0.010121 sec","job_status":"done","queue_duration_s":1.105989,"scheduling_latency_s":1.106002,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010121,"completed_at":"2026-09-03T10:15:24.840Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 146.296965] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad2e4159ea9a8f1a721def58","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad2e4159ea9a8f1a721def58: start","job_status":"start","queue_duration_s":1.116944,"scheduling_latency_s":1.116956}1442gitlab # [ 146.308773] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad2e4159ea9a8f1a721def58","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad2e4159ea9a8f1a721def58: done: 0.00963 sec","job_status":"done","queue_duration_s":1.116944,"scheduling_latency_s":1.116956,"redis_calls":13,"redis_duration_s":0.0016279999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00963,"completed_at":"2026-09-03T10:15:24.850Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 146.350312] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d54c6998c76147566383f4f2","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d54c6998c76147566383f4f2: start","job_status":"start","queue_duration_s":1.127409,"scheduling_latency_s":1.127422}1444gitlab # [ 146.354607] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d54c6998c76147566383f4f2","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d54c6998c76147566383f4f2: done: 0.009703 sec","job_status":"done","queue_duration_s":1.127409,"scheduling_latency_s":1.127422,"redis_calls":13,"redis_duration_s":0.00163,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009703,"completed_at":"2026-09-03T10:15:24.861Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 146.400814] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc16149f87e500bd92223323","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc16149f87e500bd92223323: start","job_status":"start","queue_duration_s":1.137944,"scheduling_latency_s":1.137957}1446gitlab # [ 146.401821] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc16149f87e500bd92223323","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc16149f87e500bd92223323: done: 0.009607 sec","job_status":"done","queue_duration_s":1.137944,"scheduling_latency_s":1.137957,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009607,"completed_at":"2026-09-03T10:15:24.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 146.452684] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e27f693781c9f7af4f16434f","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e27f693781c9f7af4f16434f: start","job_status":"start","queue_duration_s":1.148568,"scheduling_latency_s":1.148581}1448gitlab # [ 146.453279] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e27f693781c9f7af4f16434f","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e27f693781c9f7af4f16434f: done: 0.009581 sec","job_status":"done","queue_duration_s":1.148568,"scheduling_latency_s":1.148581,"redis_calls":13,"redis_duration_s":0.001598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009581,"completed_at":"2026-09-03T10:15:24.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 146.503854] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39f877c2994784021d39e3ac","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39f877c2994784021d39e3ac: start","job_status":"start","queue_duration_s":1.15911,"scheduling_latency_s":1.159123}1450gitlab # [ 146.504663] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39f877c2994784021d39e3ac","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39f877c2994784021d39e3ac: done: 0.009572 sec","job_status":"done","queue_duration_s":1.15911,"scheduling_latency_s":1.159123,"redis_calls":13,"redis_duration_s":0.001701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009572,"completed_at":"2026-09-03T10:15:24.893Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 146.556840] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b56e81ea3466579d25e52fa6","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b56e81ea3466579d25e52fa6: start","job_status":"start","queue_duration_s":1.169656,"scheduling_latency_s":1.169668}1452gitlab # [ 146.557650] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b56e81ea3466579d25e52fa6","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b56e81ea3466579d25e52fa6: done: 0.009614 sec","job_status":"done","queue_duration_s":1.169656,"scheduling_latency_s":1.169668,"redis_calls":13,"redis_duration_s":0.001739,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009614,"completed_at":"2026-09-03T10:15:24.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 146.598053] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e35010116a89c85f7c91ad78","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e35010116a89c85f7c91ad78: start","job_status":"start","queue_duration_s":1.180384,"scheduling_latency_s":1.180397}1454gitlab # [ 146.598781] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e35010116a89c85f7c91ad78","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e35010116a89c85f7c91ad78: done: 0.010096 sec","job_status":"done","queue_duration_s":1.180384,"scheduling_latency_s":1.180397,"redis_calls":13,"redis_duration_s":0.00164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010096,"completed_at":"2026-09-03T10:15:24.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 146.615628] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26183a151ca3e93601f07a75","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26183a151ca3e93601f07a75: start","job_status":"start","queue_duration_s":1.191475,"scheduling_latency_s":1.191489}1456gitlab # [ 146.626740] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26183a151ca3e93601f07a75","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26183a151ca3e93601f07a75: done: 0.00954 sec","job_status":"done","queue_duration_s":1.191475,"scheduling_latency_s":1.191489,"redis_calls":13,"redis_duration_s":0.00163,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00954,"completed_at":"2026-09-03T10:15:24.925Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 146.670291] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"403276238e67e43d6e453292","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-403276238e67e43d6e453292: start","job_status":"start","queue_duration_s":1.202045,"scheduling_latency_s":1.202068}1458gitlab # [ 146.671587] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"403276238e67e43d6e453292","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-403276238e67e43d6e453292: done: 0.009583 sec","job_status":"done","queue_duration_s":1.202045,"scheduling_latency_s":1.202068,"redis_calls":13,"redis_duration_s":0.0016950000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009583,"completed_at":"2026-09-03T10:15:24.935Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 146.718918] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fc1dcd6a703c50933af3023","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fc1dcd6a703c50933af3023: start","job_status":"start","queue_duration_s":1.212599,"scheduling_latency_s":1.212613}1460gitlab # [ 146.719627] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fc1dcd6a703c50933af3023","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fc1dcd6a703c50933af3023: done: 0.009763 sec","job_status":"done","queue_duration_s":1.212599,"scheduling_latency_s":1.212613,"redis_calls":13,"redis_duration_s":0.001732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009763,"completed_at":"2026-09-03T10:15:24.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 146.749645] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a37a407f7386e520a5614660","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a37a407f7386e520a5614660: start","job_status":"start","queue_duration_s":1.22325,"scheduling_latency_s":1.223263}1462gitlab # [ 146.787777] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a37a407f7386e520a5614660","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a37a407f7386e520a5614660: done: 0.009562 sec","job_status":"done","queue_duration_s":1.22325,"scheduling_latency_s":1.223263,"redis_calls":13,"redis_duration_s":0.001687,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009562,"completed_at":"2026-09-03T10:15:24.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 146.825169] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"754419c4dedd7b8408999946","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-754419c4dedd7b8408999946: start","job_status":"start","queue_duration_s":1.233613,"scheduling_latency_s":1.233626}1464gitlab # [ 146.825920] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"754419c4dedd7b8408999946","created_at":"2026-09-03T10:15:23.583Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-634a7f1dcaee40e9","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"48f99e9780cb0fe15f5fa3a3b82c964f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.724Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-754419c4dedd7b8408999946: done: 0.009646 sec","job_status":"done","queue_duration_s":1.233613,"scheduling_latency_s":1.233626,"redis_calls":13,"redis_duration_s":0.0015270000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009646,"completed_at":"2026-09-03T10:15:24.967Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 146.836992] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"40cfad4c693e8742034df468","created_at":"2026-09-03T10:15:23.719Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b2aa2dbf296b2b4acc821b73342d054","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.725Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-40cfad4c693e8742034df468: start","job_status":"start","queue_duration_s":1.242818,"scheduling_latency_s":1.242831}1466gitlab # [ 146.849242] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.986Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"40cfad4c693e8742034df468","created_at":"2026-09-03T10:15:23.719Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b2aa2dbf296b2b4acc821b73342d054","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.725Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-40cfad4c693e8742034df468: done: 0.018348 sec","job_status":"done","queue_duration_s":1.242818,"scheduling_latency_s":1.242831,"redis_calls":10,"redis_duration_s":0.001397,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000326,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000802,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227404,"duration_s":0.018348,"completed_at":"2026-09-03T10:15:24.986Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002112,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 146.903685] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:24.987Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b9e9145b6e6ad5c31119508d","created_at":"2026-09-03T10:15:23.734Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6836f616bd6d80689af1157363afb7b0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.790Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b9e9145b6e6ad5c31119508d: start","job_status":"start","queue_duration_s":1.197099,"scheduling_latency_s":1.197114}1468gitlab # [ 146.904580] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b9e9145b6e6ad5c31119508d","created_at":"2026-09-03T10:15:23.734Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6836f616bd6d80689af1157363afb7b0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.790Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b9e9145b6e6ad5c31119508d: done: 0.013587 sec","job_status":"done","queue_duration_s":1.197099,"scheduling_latency_s":1.197114,"redis_calls":12,"redis_duration_s":0.001535,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000277,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215949,"duration_s":0.013587,"completed_at":"2026-09-03T10:15:25.001Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 146.934656] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a271eb2045ea429c436f8fa6","created_at":"2026-09-03T10:15:23.795Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a06eb10efef3f8918c5bad54b493a8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.805Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a271eb2045ea429c436f8fa6: start","job_status":"start","queue_duration_s":1.196413,"scheduling_latency_s":1.196426}1470gitlab # [ 146.981374] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.009Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1471gitlab # [ 146.982138] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.014Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"a271eb2045ea429c436f8fa6","created_at":"2026-09-03T10:15:23.795Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a06eb10efef3f8918c5bad54b493a8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.805Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-a271eb2045ea429c436f8fa6: done: 0.012255 sec","job_status":"done","queue_duration_s":1.196413,"scheduling_latency_s":1.196426,"redis_calls":10,"redis_duration_s":0.0015500000000000002,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000444749,"duration_s":0.012255,"completed_at":"2026-09-03T10:15:25.014Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000484,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 146.986249] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.015Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0070ddc4b3aa309e8958c63f","created_at":"2026-09-03T10:15:23.816Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0202bd5d91b83455910b54f25bf3b054","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.822Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0070ddc4b3aa309e8958c63f: start","job_status":"start","queue_duration_s":1.193191,"scheduling_latency_s":1.193204}1473gitlab # [ 147.028948] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0070ddc4b3aa309e8958c63f","created_at":"2026-09-03T10:15:23.816Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0202bd5d91b83455910b54f25bf3b054","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.822Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0070ddc4b3aa309e8958c63f: done: 0.270802 sec","job_status":"done","queue_duration_s":1.193191,"scheduling_latency_s":1.193204,"redis_calls":565,"redis_duration_s":0.065299,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000245,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.064272,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.220978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002288,"duration_s":0.270802,"completed_at":"2026-09-03T10:15:25.286Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 147.034078] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.287Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"3f3ca70e3506312b3a73ddc4","created_at":"2026-09-03T10:15:23.841Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34faab00c1081469bd6ca7a6c1a6d4e7","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.847Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-3f3ca70e3506312b3a73ddc4: start","job_status":"start","queue_duration_s":1.440357,"scheduling_latency_s":1.440372}1475gitlab # [ 147.045631] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.304Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"3f3ca70e3506312b3a73ddc4","created_at":"2026-09-03T10:15:23.841Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34faab00c1081469bd6ca7a6c1a6d4e7","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.847Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-3f3ca70e3506312b3a73ddc4: done: 0.017184 sec","job_status":"done","queue_duration_s":1.440357,"scheduling_latency_s":1.440372,"redis_calls":10,"redis_duration_s":0.0013080000000000001,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020254,"duration_s":0.017184,"completed_at":"2026-09-03T10:15:25.304Z","load_balancing_strategy":"primary","db_duration_s":0.002461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 147.090246] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.305Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9b66fc99a7bf62a25c478ee5","created_at":"2026-09-03T10:15:23.854Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3c656b76987d91f23cd72c039e5980f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.860Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9b66fc99a7bf62a25c478ee5: start","job_status":"start","queue_duration_s":1.445142,"scheduling_latency_s":1.445154}1477gitlab # [ 147.130054] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9b66fc99a7bf62a25c478ee5","created_at":"2026-09-03T10:15:23.854Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3c656b76987d91f23cd72c039e5980f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.860Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9b66fc99a7bf62a25c478ee5: done: 0.013446 sec","job_status":"done","queue_duration_s":1.445142,"scheduling_latency_s":1.445154,"redis_calls":12,"redis_duration_s":0.0015789999999999999,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000547,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000305346,"exclusive_lock_hold_duration_s":0.001787657,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.013446,"completed_at":"2026-09-03T10:15:25.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000881,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 147.133945] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f3a10d306c8899c094675a30","created_at":"2026-09-03T10:15:23.867Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4e25d4b5b17470ed017ce04af9adf2a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:15:23.872Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f3a10d306c8899c094675a30: start","job_status":"start","queue_duration_s":1.447605,"scheduling_latency_s":1.447618}1479gitlab # [ 147.145752] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.326Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1480gitlab # [ 147.148727] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.330Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f3a10d306c8899c094675a30","created_at":"2026-09-03T10:15:23.867Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4e25d4b5b17470ed017ce04af9adf2a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:15:23.872Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f3a10d306c8899c094675a30: done: 0.01084 sec","job_status":"done","queue_duration_s":1.447605,"scheduling_latency_s":1.447618,"redis_calls":10,"redis_duration_s":0.001336,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021148,"duration_s":0.01084,"completed_at":"2026-09-03T10:15:25.330Z","load_balancing_strategy":"primary","db_duration_s":0.000227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 147.205957] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.331Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","jid":"98fb78922c0dbc3dffbe0d95","created_at":"2026-09-03T10:15:23.880Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21f3b3d96fffe2125c7741b359ec1643","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.886Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::StuckProjectImportJobsWorker JID-98fb78922c0dbc3dffbe0d95: start","job_status":"start","queue_duration_s":1.44489,"scheduling_latency_s":1.444903}1482gitlab # [ 147.206697] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.347Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::StuckProjectImportJobsWorker","jid":"98fb78922c0dbc3dffbe0d95","created_at":"2026-09-03T10:15:23.880Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21f3b3d96fffe2125c7741b359ec1643","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:65d1a1bda9b2a0115065de4f20868193e50a174305a7166fa20fbc82759e43e4","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.886Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::StuckProjectImportJobsWorker JID-98fb78922c0dbc3dffbe0d95: done: 0.016109 sec","job_status":"done","queue_duration_s":1.44489,"scheduling_latency_s":1.444903,"redis_calls":10,"redis_duration_s":0.001305,"redis_read_bytes":9,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":236,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205054,"duration_s":0.016109,"completed_at":"2026-09-03T10:15:25.347Z","load_balancing_strategy":"primary","db_duration_s":0.00214,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 147.246555] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.348Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"811d50439f4231899a3f878b","created_at":"2026-09-03T10:15:23.955Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"605f45dba03c9a7f481be2cf04ed41d5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.976Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-811d50439f4231899a3f878b: start","job_status":"start","queue_duration_s":1.371663,"scheduling_latency_s":1.371677}1484gitlab # [ 147.247294] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.408Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"811d50439f4231899a3f878b","created_at":"2026-09-03T10:15:23.955Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"605f45dba03c9a7f481be2cf04ed41d5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:23.976Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-811d50439f4231899a3f878b: done: 0.060233 sec","job_status":"done","queue_duration_s":1.371663,"scheduling_latency_s":1.371677,"redis_calls":12,"redis_duration_s":0.0015639999999999999,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000569,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000330489,"exclusive_lock_hold_duration_s":0.003847137,"duration_s":0.060233,"completed_at":"2026-09-03T10:15:25.408Z","load_balancing_strategy":"primary","db_duration_s":0.001592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 147.297777] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c73ed0b890f217416a824a11","created_at":"2026-09-03T10:15:23.996Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"286f2306bbde616aaf88838acce8aa89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.012Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c73ed0b890f217416a824a11: start","job_status":"start","queue_duration_s":1.398027,"scheduling_latency_s":1.398043}1486gitlab # [ 147.298651] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c73ed0b890f217416a824a11","created_at":"2026-09-03T10:15:23.996Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"286f2306bbde616aaf88838acce8aa89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.012Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c73ed0b890f217416a824a11: done: 0.011785 sec","job_status":"done","queue_duration_s":1.398027,"scheduling_latency_s":1.398043,"redis_calls":10,"redis_duration_s":0.001308,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212596,"duration_s":0.011785,"completed_at":"2026-09-03T10:15:25.421Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 147.327863] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"3f31564de3658a1ca685da2b","created_at":"2026-09-03T10:15:24.034Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.046Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-3f31564de3658a1ca685da2b: start","job_status":"start","queue_duration_s":1.376368,"scheduling_latency_s":1.376381}1488gitlab # [ 147.369075] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.443Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"3f31564de3658a1ca685da2b","created_at":"2026-09-03T10:15:24.034Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.046Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-3f31564de3658a1ca685da2b: done: 0.020817 sec","job_status":"done","queue_duration_s":1.376368,"scheduling_latency_s":1.376381,"redis_calls":19,"redis_duration_s":0.002497,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000519,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000173,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001336,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000231873,"duration_s":0.020817,"completed_at":"2026-09-03T10:15:25.443Z","load_balancing_strategy":"primary","db_duration_s":0.001154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 147.377054] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"e5d7885200697710a8bbcb30","created_at":"2026-09-03T10:15:24.063Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83184c2794ff98c9837215e30da50267","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.075Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-e5d7885200697710a8bbcb30: start","job_status":"start","queue_duration_s":1.368615,"scheduling_latency_s":1.368628}1490gitlab # [ 147.389063] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.460Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"e5d7885200697710a8bbcb30","created_at":"2026-09-03T10:15:24.063Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83184c2794ff98c9837215e30da50267","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.075Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-e5d7885200697710a8bbcb30: done: 0.015599 sec","job_status":"done","queue_duration_s":1.368615,"scheduling_latency_s":1.368628,"redis_calls":10,"redis_duration_s":0.001297,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239416,"duration_s":0.015599,"completed_at":"2026-09-03T10:15:25.460Z","load_balancing_strategy":"primary","db_duration_s":0.002072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 147.458775] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.460Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"085fccadaeb7ebec01fccab6","created_at":"2026-09-03T10:15:24.092Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d84414d60710d8bafd43c4aa1a7a1cd7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.111Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-085fccadaeb7ebec01fccab6: start","job_status":"start","queue_duration_s":1.349648,"scheduling_latency_s":1.349661}1492gitlab # [ 147.459643] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"085fccadaeb7ebec01fccab6","created_at":"2026-09-03T10:15:24.092Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d84414d60710d8bafd43c4aa1a7a1cd7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.111Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-085fccadaeb7ebec01fccab6: done: 0.13182 sec","job_status":"done","queue_duration_s":1.349648,"scheduling_latency_s":1.349661,"redis_calls":12,"redis_duration_s":0.001742,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.024,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.100451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00030786,"exclusive_lock_hold_duration_s":0.119015685,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.13182,"completed_at":"2026-09-03T10:15:25.592Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035339,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 147.507990] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"cc06265bba2009eb14e9d620","created_at":"2026-09-03T10:15:24.126Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c20d08bc5c1d0c78d198a48d7533ba0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.133Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-cc06265bba2009eb14e9d620: start","job_status":"start","queue_duration_s":1.460417,"scheduling_latency_s":1.460431}1494gitlab # [ 147.508813] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.620Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"cc06265bba2009eb14e9d620","created_at":"2026-09-03T10:15:24.126Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c20d08bc5c1d0c78d198a48d7533ba0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.133Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-cc06265bba2009eb14e9d620: done: 0.026586 sec","job_status":"done","queue_duration_s":1.460417,"scheduling_latency_s":1.460431,"redis_calls":10,"redis_duration_s":0.0013,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212318,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.026586,"completed_at":"2026-09-03T10:15:25.620Z","load_balancing_strategy":"primary","db_duration_s":0.012758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 147.527517] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"05cfaf3e4e88d9d998458f47","created_at":"2026-09-03T10:15:24.144Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"161ef4be3d9959a63b29aeca163f9a93","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.147Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-05cfaf3e4e88d9d998458f47: start","job_status":"start","queue_duration_s":1.473906,"scheduling_latency_s":1.473919}1496gitlab # [ 147.540758] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"05cfaf3e4e88d9d998458f47","created_at":"2026-09-03T10:15:24.144Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"161ef4be3d9959a63b29aeca163f9a93","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.147Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-05cfaf3e4e88d9d998458f47: done: 0.011831 sec","job_status":"done","queue_duration_s":1.473906,"scheduling_latency_s":1.473919,"redis_calls":11,"redis_duration_s":0.001413,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215111,"duration_s":0.011831,"completed_at":"2026-09-03T10:15:25.633Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000956,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 147.602459] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.634Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMigrateExternalDiffsWorker","jid":"7bd07494d063dc93a7a98d6e","created_at":"2026-09-03T10:15:24.156Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19912fa75da59b8a206c79877f8a0b8e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.172Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMigrateExternalDiffsWorker JID-7bd07494d063dc93a7a98d6e: start","job_status":"start","queue_duration_s":1.461394,"scheduling_latency_s":1.461408}1498gitlab # [ 147.603208] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.646Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMigrateExternalDiffsWorker","jid":"7bd07494d063dc93a7a98d6e","created_at":"2026-09-03T10:15:24.156Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19912fa75da59b8a206c79877f8a0b8e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1f56d9ee4ddd7fd390ae51dd41358b9c806acf2491cad4e28b1956ebf69a9b52","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.172Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMigrateExternalDiffsWorker JID-7bd07494d063dc93a7a98d6e: done: 0.012507 sec","job_status":"done","queue_duration_s":1.461394,"scheduling_latency_s":1.461408,"redis_calls":12,"redis_duration_s":0.0015929999999999998,"redis_read_bytes":11,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000294,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":311,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000551,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":533,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000305626,"exclusive_lock_hold_duration_s":0.000299759,"duration_s":0.012507,"completed_at":"2026-09-03T10:15:25.646Z","load_balancing_strategy":"primary","db_duration_s":0.000822,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 147.627579] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8ed9393b3ede25ad4213ca15","created_at":"2026-09-03T10:15:24.189Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9649d068f84992612f87230325a23acc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:15:24.210Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8ed9393b3ede25ad4213ca15: start","job_status":"start","queue_duration_s":1.437078,"scheduling_latency_s":1.437091}1500gitlab # [ 147.639259] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8ed9393b3ede25ad4213ca15","created_at":"2026-09-03T10:15:24.189Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9649d068f84992612f87230325a23acc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:15:24.210Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8ed9393b3ede25ad4213ca15: done: 0.018323 sec","job_status":"done","queue_duration_s":1.437078,"scheduling_latency_s":1.437091,"redis_calls":10,"redis_duration_s":0.001305,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000262,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222095,"duration_s":0.018323,"completed_at":"2026-09-03T10:15:25.665Z","load_balancing_strategy":"primary","db_duration_s":0.002277,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 147.678572] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9e7ef96a5900819e4345e2f6","created_at":"2026-09-03T10:15:24.218Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8bc83dad7588170712b3889ee0ef6ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.233Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9e7ef96a5900819e4345e2f6: start","job_status":"start","queue_duration_s":1.43265,"scheduling_latency_s":1.432662}1502gitlab # [ 147.687943] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.673Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1503gitlab # [ 147.691312] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.677Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9e7ef96a5900819e4345e2f6","created_at":"2026-09-03T10:15:24.218Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8bc83dad7588170712b3889ee0ef6ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:24.233Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9e7ef96a5900819e4345e2f6: done: 0.011303 sec","job_status":"done","queue_duration_s":1.43265,"scheduling_latency_s":1.432662,"redis_calls":10,"redis_duration_s":0.001207,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000243,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000238858,"duration_s":0.011303,"completed_at":"2026-09-03T10:15:25.677Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000234,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504gitlab # [ 147.746157] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.678Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"96cfe864b17129fa64d71245","created_at":"2026-09-03T10:15:25.431Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-055a7fef1f594424","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:25.438Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-96cfe864b17129fa64d71245: start","job_status":"start","queue_duration_s":0.240048,"scheduling_latency_s":0.240061}1505gitlab # [ 147.746888] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.689Z","message":"No existing merge request to be cleaned up."}1506gitlab # [ 147.747571] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.698Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"96cfe864b17129fa64d71245","created_at":"2026-09-03T10:15:25.431Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-055a7fef1f594424","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:25.438Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-96cfe864b17129fa64d71245: done: 0.019615 sec","job_status":"done","queue_duration_s":0.240048,"scheduling_latency_s":0.240061,"redis_calls":13,"redis_duration_s":0.001739,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000866,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208406,"duration_s":0.019615,"completed_at":"2026-09-03T10:15:25.698Z","load_balancing_strategy":"primary","db_duration_s":0.005185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 147.777690] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.699Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ed5bd707812abf1e1d22faab","created_at":"2026-09-03T10:15:25.431Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-055a7fef1f594424","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:25.438Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ed5bd707812abf1e1d22faab: start","job_status":"start","queue_duration_s":0.260578,"scheduling_latency_s":0.260591}1508gitlab # [ 147.821854] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.705Z","message":"No existing merge request to be cleaned up."}1509gitlab # [ 147.822640] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.709Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ed5bd707812abf1e1d22faab","created_at":"2026-09-03T10:15:25.431Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-055a7fef1f594424","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:25.438Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ed5bd707812abf1e1d22faab: done: 0.010558 sec","job_status":"done","queue_duration_s":0.260578,"scheduling_latency_s":0.260591,"redis_calls":10,"redis_duration_s":0.001276,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010558,"completed_at":"2026-09-03T10:15:25.709Z","load_balancing_strategy":"primary","db_duration_s":0.000988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510gitlab # [ 147.823655] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.710Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a61430546e9e47f8c266988a","created_at":"2026-09-03T10:15:25.431Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-055a7fef1f594424","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:25.438Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a61430546e9e47f8c266988a: start","job_status":"start","queue_duration_s":0.271968,"scheduling_latency_s":0.271986}1511gitlab # [ 147.860683] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.717Z","message":"No existing merge request to be cleaned up."}1512gitlab # [ 147.861553] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.721Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a61430546e9e47f8c266988a","created_at":"2026-09-03T10:15:25.431Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-055a7fef1f594424","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:25.438Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a61430546e9e47f8c266988a: done: 0.010888 sec","job_status":"done","queue_duration_s":0.271968,"scheduling_latency_s":0.271986,"redis_calls":10,"redis_duration_s":0.0012850000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000741,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010888,"completed_at":"2026-09-03T10:15:25.721Z","load_balancing_strategy":"primary","db_duration_s":0.000968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 147.898731] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.722Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0e0d534377124c61ffef2b8f","created_at":"2026-09-03T10:15:25.431Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-055a7fef1f594424","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:25.438Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0e0d534377124c61ffef2b8f: start","job_status":"start","queue_duration_s":0.283663,"scheduling_latency_s":0.283892}1514gitlab # [ 147.899453] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.728Z","message":"No existing merge request to be cleaned up."}1515gitlab # [ 147.899926] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:15:25.736Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0e0d534377124c61ffef2b8f","created_at":"2026-09-03T10:15:25.431Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-055a7fef1f594424","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"c4fc78661fbf7519af8ca495171eb5cf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:15:25.438Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0e0d534377124c61ffef2b8f: done: 0.01383 sec","job_status":"done","queue_duration_s":0.283663,"scheduling_latency_s":0.283892,"redis_calls":10,"redis_duration_s":0.0013540000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00084,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01383,"completed_at":"2026-09-03T10:15:25.736Z","load_balancing_strategy":"primary","db_duration_s":0.00374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516gitlab # [ 148.098301] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 3685779635 wd_nsec: 36857794021517gitlab # [ 161.345995] setup-gitlab-start[2210]: W, [2026-09-03T10:15:41.334219 #2210] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1518gitlab # [ 182.084592] postgres[2411]: [2411] ERROR: relation "table_without_models" does not exist1519gitlab # [ 182.087553] postgres[2411]: [2411] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1520gitlab # [ 182.235949] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:02.223Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"fd6cf1c61c7f07bde2ce6392","created_at":"2026-09-03T10:16:02.171Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"99f19e08d1cbcd2686faf552834cdc9b","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.219Z","job_size_bytes":3,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-fd6cf1c61c7f07bde2ce6392: start","job_status":"start","queue_duration_s":0.004352,"scheduling_latency_s":0.004407}1521gitlab # [ 182.417826] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:02.406Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"fd6cf1c61c7f07bde2ce6392","created_at":"2026-09-03T10:16:02.171Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"99f19e08d1cbcd2686faf552834cdc9b","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.219Z","job_size_bytes":3,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-fd6cf1c61c7f07bde2ce6392: done: 0.182608 sec","job_status":"done","queue_duration_s":0.004352,"scheduling_latency_s":0.004407,"redis_calls":11,"redis_duration_s":0.029878000000000002,"redis_read_bytes":13,"redis_write_bytes":1207,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.007686,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018824,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001242337,"duration_s":0.182608,"completed_at":"2026-09-03T10:16:02.406Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037093,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 182.493544] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:02.482Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"a1bd659e281377a8f0878aab","created_at":"2026-09-03T10:16:02.260Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.274Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-a1bd659e281377a8f0878aab: start","job_status":"start","queue_duration_s":0.207013,"scheduling_latency_s":0.207054}1523gitlab # [ 183.064176] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.052Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"a1bd659e281377a8f0878aab","created_at":"2026-09-03T10:16:02.260Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.274Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-a1bd659e281377a8f0878aab: done: 0.570433 sec","job_status":"done","queue_duration_s":0.207013,"scheduling_latency_s":0.207054,"redis_calls":65,"redis_duration_s":0.177334,"redis_read_bytes":114,"redis_write_bytes":41332,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.008043,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.002382,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37035,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.151382,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.015527,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.131165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003216889,"duration_s":0.570433,"completed_at":"2026-09-03T10:16:03.052Z","load_balancing_strategy":"primary","db_duration_s":0.021708,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524gitlab # [ 183.135094] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.123Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"7ce2ba2bced68f9df29a5ebd","created_at":"2026-09-03T10:16:02.285Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b02313c206d7315999c8cf8d2ade81f","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.306Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-7ce2ba2bced68f9df29a5ebd: start","job_status":"start","queue_duration_s":0.81675,"scheduling_latency_s":0.816773}1525gitlab # [ 183.208073] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.196Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"7ce2ba2bced68f9df29a5ebd","created_at":"2026-09-03T10:16:02.285Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b02313c206d7315999c8cf8d2ade81f","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.306Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-7ce2ba2bced68f9df29a5ebd: done: 0.072742 sec","job_status":"done","queue_duration_s":0.81675,"scheduling_latency_s":0.816773,"redis_calls":12,"redis_duration_s":0.013045000000000001,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.002285,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002882,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001669,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001369727,"duration_s":0.072742,"completed_at":"2026-09-03T10:16:03.196Z","load_balancing_strategy":"primary","db_duration_s":0.002276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 183.210680] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a0fe4e0061e72bfb44d49673","created_at":"2026-09-03T10:16:02.319Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.324Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a0fe4e0061e72bfb44d49673: start","job_status":"start","queue_duration_s":0.874248,"scheduling_latency_s":0.874274}1527gitlab # [ 183.269290] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a0fe4e0061e72bfb44d49673","created_at":"2026-09-03T10:16:02.319Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.324Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a0fe4e0061e72bfb44d49673: done: 0.059032 sec","job_status":"done","queue_duration_s":0.874248,"scheduling_latency_s":0.874274,"redis_calls":60,"redis_duration_s":0.008292,"redis_read_bytes":109,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.000422,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.007306,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000564,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059032,"completed_at":"2026-09-03T10:16:03.257Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528gitlab # [ 183.271366] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3a7fa46c878b568a7843ef6d","created_at":"2026-09-03T10:16:02.332Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d77e2cf8e7f5e27fbc800af075f8106d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.336Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3a7fa46c878b568a7843ef6d: start","job_status":"start","queue_duration_s":0.922197,"scheduling_latency_s":0.922214}1529gitlab # [ 183.385349] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3a7fa46c878b568a7843ef6d","created_at":"2026-09-03T10:16:02.332Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d77e2cf8e7f5e27fbc800af075f8106d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.336Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3a7fa46c878b568a7843ef6d: done: 0.115491 sec","job_status":"done","queue_duration_s":0.922197,"scheduling_latency_s":0.922214,"redis_calls":9,"redis_duration_s":0.002685,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001229,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.046,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000318197,"exclusive_lock_hold_duration_s":0.104529486,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.115491,"completed_at":"2026-09-03T10:16:03.373Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.098873,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530gitlab # [ 183.387195] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2fd708a5f50658f7296777fc","created_at":"2026-09-03T10:16:02.342Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d17df9efb44715aa51f25cd048cc27b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.350Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2fd708a5f50658f7296777fc: start","job_status":"start","queue_duration_s":1.024834,"scheduling_latency_s":1.02485}1531gitlab # [ 183.400489] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2fd708a5f50658f7296777fc","created_at":"2026-09-03T10:16:02.342Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d17df9efb44715aa51f25cd048cc27b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.350Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2fd708a5f50658f7296777fc: done: 0.01375 sec","job_status":"done","queue_duration_s":1.024834,"scheduling_latency_s":1.02485,"redis_calls":11,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":136,"redis_write_bytes":1288,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000121,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004233219,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.01375,"completed_at":"2026-09-03T10:16:03.389Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002325,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1532gitlab # [ 183.443902] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"390191fa68a8a42697ab0913","created_at":"2026-09-03T10:16:02.362Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60ad5d611daeeba52340b7ea2c8164c2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.370Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-390191fa68a8a42697ab0913: start","job_status":"start","queue_duration_s":1.019511,"scheduling_latency_s":1.019524}1533gitlab # [ 183.452944] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.394Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1534gitlab # [ 183.456521] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.397Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"390191fa68a8a42697ab0913","created_at":"2026-09-03T10:16:02.362Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60ad5d611daeeba52340b7ea2c8164c2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.370Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-390191fa68a8a42697ab0913: done: 0.007713 sec","job_status":"done","queue_duration_s":1.019511,"scheduling_latency_s":1.019524,"redis_calls":7,"redis_duration_s":0.00101,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007713,"completed_at":"2026-09-03T10:16:03.397Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 183.503830] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.398Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c3656679843a36fcd8875ff8","created_at":"2026-09-03T10:16:02.397Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f870b6182377157231ac735e79235ce7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.400Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c3656679843a36fcd8875ff8: start","job_status":"start","queue_duration_s":0.997702,"scheduling_latency_s":0.997714}1536gitlab # [ 183.506739] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.465Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c3656679843a36fcd8875ff8","created_at":"2026-09-03T10:16:02.397Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f870b6182377157231ac735e79235ce7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.400Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c3656679843a36fcd8875ff8: done: 0.066848 sec","job_status":"done","queue_duration_s":0.997702,"scheduling_latency_s":0.997714,"redis_calls":7,"redis_duration_s":0.001013,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.056,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066848,"completed_at":"2026-09-03T10:16:03.465Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056022,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 183.563728] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d858b9c4a06ca442661a86c6","created_at":"2026-09-03T10:16:02.479Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa1c4a080f7c4118244ae8441d724c9f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:16:02.499Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d858b9c4a06ca442661a86c6: start","job_status":"start","queue_duration_s":0.966748,"scheduling_latency_s":0.966768}1538gitlab # [ 183.564866] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.471Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1539gitlab # [ 183.567775] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.474Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d858b9c4a06ca442661a86c6","created_at":"2026-09-03T10:16:02.479Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa1c4a080f7c4118244ae8441d724c9f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:16:02.499Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d858b9c4a06ca442661a86c6: done: 0.007903 sec","job_status":"done","queue_duration_s":0.966748,"scheduling_latency_s":0.966768,"redis_calls":7,"redis_duration_s":0.0009400000000000001,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007903,"completed_at":"2026-09-03T10:16:03.474Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540gitlab # [ 183.622307] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.475Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0f7e021233ed56a5c38b7db4","created_at":"2026-09-03T10:16:02.521Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56688240b09558aadb5221b15470383b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.526Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0f7e021233ed56a5c38b7db4: start","job_status":"start","queue_duration_s":0.948933,"scheduling_latency_s":0.948945}1541gitlab # [ 183.623296] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.479Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1542gitlab # [ 183.624060] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.482Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0f7e021233ed56a5c38b7db4","created_at":"2026-09-03T10:16:02.521Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56688240b09558aadb5221b15470383b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.526Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0f7e021233ed56a5c38b7db4: done: 0.007522 sec","job_status":"done","queue_duration_s":0.948933,"scheduling_latency_s":0.948945,"redis_calls":7,"redis_duration_s":0.0009159999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007522,"completed_at":"2026-09-03T10:16:03.482Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 183.691057] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.483Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"844c024011bfb06e8ce14ba9","created_at":"2026-09-03T10:16:02.597Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56231ad28b0e1379181d420dcbc2b14e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.607Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-844c024011bfb06e8ce14ba9: start","job_status":"start","queue_duration_s":0.876026,"scheduling_latency_s":0.876038}1544gitlab # [ 183.692307] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"844c024011bfb06e8ce14ba9","created_at":"2026-09-03T10:16:02.597Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56231ad28b0e1379181d420dcbc2b14e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.607Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-844c024011bfb06e8ce14ba9: done: 0.020446 sec","job_status":"done","queue_duration_s":0.876026,"scheduling_latency_s":0.876038,"redis_calls":9,"redis_duration_s":0.0013020000000000002,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00064,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000367365,"exclusive_lock_hold_duration_s":0.008568408,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.020446,"completed_at":"2026-09-03T10:16:03.504Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007377,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 183.710205] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"96c1809d243cae61b1d5061c","created_at":"2026-09-03T10:16:02.629Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac60602873e934e24c012258e34923b7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.638Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-96c1809d243cae61b1d5061c: start","job_status":"start","queue_duration_s":0.866929,"scheduling_latency_s":0.866942}1546gitlab # [ 183.723863] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"96c1809d243cae61b1d5061c","created_at":"2026-09-03T10:16:02.629Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac60602873e934e24c012258e34923b7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.638Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-96c1809d243cae61b1d5061c: done: 0.008638 sec","job_status":"done","queue_duration_s":0.866929,"scheduling_latency_s":0.866942,"redis_calls":7,"redis_duration_s":0.000892,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008638,"completed_at":"2026-09-03T10:16:03.513Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 183.777713] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.514Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7a456e792bafe5a8808e2a8b","created_at":"2026-09-03T10:16:02.652Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e215da9eb3dce0b218e133ebb3f45789","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.657Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7a456e792bafe5a8808e2a8b: start","job_status":"start","queue_duration_s":0.856991,"scheduling_latency_s":0.857003}1548gitlab # [ 183.778748] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7a456e792bafe5a8808e2a8b","created_at":"2026-09-03T10:16:02.652Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e215da9eb3dce0b218e133ebb3f45789","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.657Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7a456e792bafe5a8808e2a8b: done: 0.008171 sec","job_status":"done","queue_duration_s":0.856991,"scheduling_latency_s":0.857003,"redis_calls":8,"redis_duration_s":0.0011020000000000001,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000132,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008171,"completed_at":"2026-09-03T10:16:03.522Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 183.825237] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"615a6fb3736ad0726d3f02f2","created_at":"2026-09-03T10:16:02.671Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a38451c65da8ebc73e5e9ddbe54eacd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.680Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-615a6fb3736ad0726d3f02f2: start","job_status":"start","queue_duration_s":0.842977,"scheduling_latency_s":0.842991}1550gitlab # [ 183.827799] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.528Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1551gitlab # [ 183.831122] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"615a6fb3736ad0726d3f02f2","created_at":"2026-09-03T10:16:02.671Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a38451c65da8ebc73e5e9ddbe54eacd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.680Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-615a6fb3736ad0726d3f02f2: done: 0.007407 sec","job_status":"done","queue_duration_s":0.842977,"scheduling_latency_s":0.842991,"redis_calls":7,"redis_duration_s":0.0009090000000000001,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007407,"completed_at":"2026-09-03T10:16:03.531Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552gitlab # [ 183.871699] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:03.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"aa2529c15aab0107520aec49","created_at":"2026-09-03T10:16:02.698Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4b0ca161f8ca8bdcf31fbb8a177de89","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.706Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-aa2529c15aab0107520aec49: start","job_status":"start","queue_duration_s":0.825618,"scheduling_latency_s":0.825631}1553gitlab # [ 184.286521] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.274Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"aa2529c15aab0107520aec49","created_at":"2026-09-03T10:16:02.698Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4b0ca161f8ca8bdcf31fbb8a177de89","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.706Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-aa2529c15aab0107520aec49: done: 0.742858 sec","job_status":"done","queue_duration_s":0.825618,"scheduling_latency_s":0.825631,"redis_calls":562,"redis_duration_s":0.559392,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000803,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.558589,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.202506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.742858,"completed_at":"2026-09-03T10:16:04.274Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554gitlab # [ 184.327473] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.316Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"22dd6d6503d6d38d9b735dff","created_at":"2026-09-03T10:16:02.729Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82622ca57bbbb550c93a543d261cd0bb","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.737Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-22dd6d6503d6d38d9b735dff: start","job_status":"start","queue_duration_s":1.578777,"scheduling_latency_s":1.578808}1555gitlab # [ 184.363383] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"22dd6d6503d6d38d9b735dff","created_at":"2026-09-03T10:16:02.729Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82622ca57bbbb550c93a543d261cd0bb","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.737Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-22dd6d6503d6d38d9b735dff: done: 0.035622 sec","job_status":"done","queue_duration_s":1.578777,"scheduling_latency_s":1.578808,"redis_calls":9,"redis_duration_s":0.002294,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001325,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000801499,"exclusive_lock_hold_duration_s":0.003708013,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.035622,"completed_at":"2026-09-03T10:16:04.351Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556gitlab # [ 184.402259] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a2c6f88bd4a117197926ad6f","created_at":"2026-09-03T10:16:02.757Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a9f0e6d20e55814200617a011825d5f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:16:02.764Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a2c6f88bd4a117197926ad6f: start","job_status":"start","queue_duration_s":1.626071,"scheduling_latency_s":1.626139}1557gitlab # [ 184.415075] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.403Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1558gitlab # [ 184.423500] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.411Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a2c6f88bd4a117197926ad6f","created_at":"2026-09-03T10:16:02.757Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a9f0e6d20e55814200617a011825d5f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:16:02.764Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a2c6f88bd4a117197926ad6f: done: 0.021595 sec","job_status":"done","queue_duration_s":1.626071,"scheduling_latency_s":1.626139,"redis_calls":7,"redis_duration_s":0.002373,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00173,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000643,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021595,"completed_at":"2026-09-03T10:16:04.411Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559gitlab # [ 184.460728] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"392b27f9851e501a2c3d10c4","created_at":"2026-09-03T10:16:02.779Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3d8c62c5c358018a465e64aa7d299dd","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.802Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-392b27f9851e501a2c3d10c4: start","job_status":"start","queue_duration_s":1.646993,"scheduling_latency_s":1.647091}1560gitlab # [ 184.510281] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"392b27f9851e501a2c3d10c4","created_at":"2026-09-03T10:16:02.779Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3d8c62c5c358018a465e64aa7d299dd","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.802Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-392b27f9851e501a2c3d10c4: done: 0.049578 sec","job_status":"done","queue_duration_s":1.646993,"scheduling_latency_s":1.647091,"redis_calls":10,"redis_duration_s":0.002189,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000414578,"duration_s":0.049578,"completed_at":"2026-09-03T10:16:04.498Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561gitlab # [ 184.550863] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.539Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"295d7e9c1acb1d711ea8be45","created_at":"2026-09-03T10:16:02.820Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b536ef47a52b0b88c83cbf8ddf7a71f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.842Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-295d7e9c1acb1d711ea8be45: start","job_status":"start","queue_duration_s":1.696686,"scheduling_latency_s":1.696734}1562gitlab # [ 184.588651] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.576Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"295d7e9c1acb1d711ea8be45","created_at":"2026-09-03T10:16:02.820Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b536ef47a52b0b88c83cbf8ddf7a71f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.842Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-295d7e9c1acb1d711ea8be45: done: 0.037946 sec","job_status":"done","queue_duration_s":1.696686,"scheduling_latency_s":1.696734,"redis_calls":12,"redis_duration_s":0.003664,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000648,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001675,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001341,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000867429,"exclusive_lock_hold_duration_s":0.006683532,"duration_s":0.037946,"completed_at":"2026-09-03T10:16:04.576Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563gitlab # [ 184.590617] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.579Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"07ababefd05e32c205cdbab7","created_at":"2026-09-03T10:16:02.867Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ee744b4aa277ea4cb5501a060a6084c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.875Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-07ababefd05e32c205cdbab7: start","job_status":"start","queue_duration_s":1.704068,"scheduling_latency_s":1.704093}1564gitlab # [ 184.630869] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"07ababefd05e32c205cdbab7","created_at":"2026-09-03T10:16:02.867Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ee744b4aa277ea4cb5501a060a6084c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.875Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-07ababefd05e32c205cdbab7: done: 0.016938 sec","job_status":"done","queue_duration_s":1.704068,"scheduling_latency_s":1.704093,"redis_calls":9,"redis_duration_s":0.001597,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000705,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000581079,"exclusive_lock_hold_duration_s":0.003067709,"duration_s":0.016938,"completed_at":"2026-09-03T10:16:04.596Z","load_balancing_strategy":"primary","db_duration_s":0.001151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565gitlab # [ 184.673195] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.597Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"68e84387984d18d8ad15032f","created_at":"2026-09-03T10:16:02.907Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d0b30514b10f1f338ee1ee9968000c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.915Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-68e84387984d18d8ad15032f: start","job_status":"start","queue_duration_s":1.682083,"scheduling_latency_s":1.682106}1566gitlab # [ 184.673910] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"68e84387984d18d8ad15032f","created_at":"2026-09-03T10:16:02.907Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d0b30514b10f1f338ee1ee9968000c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.915Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-68e84387984d18d8ad15032f: done: 0.009253 sec","job_status":"done","queue_duration_s":1.682083,"scheduling_latency_s":1.682106,"redis_calls":7,"redis_duration_s":0.000885,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00063,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009253,"completed_at":"2026-09-03T10:16:04.606Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567gitlab # [ 184.685270] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"e673f0847286d05d88dbef20","created_at":"2026-09-03T10:16:02.934Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.940Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-e673f0847286d05d88dbef20: start","job_status":"start","queue_duration_s":1.667166,"scheduling_latency_s":1.667179}1568gitlab # [ 184.723338] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"e673f0847286d05d88dbef20","created_at":"2026-09-03T10:16:02.934Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.940Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-e673f0847286d05d88dbef20: done: 0.015202 sec","job_status":"done","queue_duration_s":1.667166,"scheduling_latency_s":1.667179,"redis_calls":14,"redis_duration_s":0.001796,"redis_read_bytes":17,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000255,"redis_queues_read_bytes":4,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015202,"completed_at":"2026-09-03T10:16:04.623Z","load_balancing_strategy":"primary","db_duration_s":0.001222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569gitlab # [ 184.770911] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8da454e0345edd7624abece0","created_at":"2026-09-03T10:16:02.964Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66734b6c21ff9a98008ce3b3d00614af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.975Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8da454e0345edd7624abece0: start","job_status":"start","queue_duration_s":1.648697,"scheduling_latency_s":1.64871}1570gitlab # [ 184.802803] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.791Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8da454e0345edd7624abece0","created_at":"2026-09-03T10:16:02.964Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66734b6c21ff9a98008ce3b3d00614af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:02.975Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8da454e0345edd7624abece0: done: 0.16742 sec","job_status":"done","queue_duration_s":1.648697,"scheduling_latency_s":1.64871,"redis_calls":11,"redis_duration_s":0.002342,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000351,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001018,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00033887,"exclusive_lock_hold_duration_s":0.155478699,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.16742,"completed_at":"2026-09-03T10:16:04.791Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053871,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571gitlab # [ 184.843175] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.831Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"440d86528583840a75aec693","created_at":"2026-09-03T10:16:02.997Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b78b416abda3ce00725df0ec583cb960","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.004Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-440d86528583840a75aec693: start","job_status":"start","queue_duration_s":1.827206,"scheduling_latency_s":1.827228}1572gitlab # [ 184.878437] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"440d86528583840a75aec693","created_at":"2026-09-03T10:16:02.997Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b78b416abda3ce00725df0ec583cb960","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.004Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-440d86528583840a75aec693: done: 0.035257 sec","job_status":"done","queue_duration_s":1.827206,"scheduling_latency_s":1.827228,"redis_calls":7,"redis_duration_s":0.0017050000000000001,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001412,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000293,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.035257,"completed_at":"2026-09-03T10:16:04.867Z","load_balancing_strategy":"primary","db_duration_s":0.008373,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573gitlab # [ 184.917364] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"47afb92df3e914e949e3c6fc","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-47afb92df3e914e949e3c6fc: start","job_status":"start","queue_duration_s":1.895724,"scheduling_latency_s":1.895754}1574gitlab # [ 184.973130] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"47afb92df3e914e949e3c6fc","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-47afb92df3e914e949e3c6fc: done: 0.055657 sec","job_status":"done","queue_duration_s":1.895724,"scheduling_latency_s":1.895754,"redis_calls":13,"redis_duration_s":0.003079,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000506,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001411,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000449778,"duration_s":0.055657,"completed_at":"2026-09-03T10:16:04.961Z","load_balancing_strategy":"primary","db_duration_s":0.006889,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575gitlab # [ 185.011120] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:04.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ff92dabef3714f489b6d4e84","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ff92dabef3714f489b6d4e84: start","job_status":"start","queue_duration_s":1.989381,"scheduling_latency_s":1.98941}1576gitlab # [ 185.071808] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ff92dabef3714f489b6d4e84","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ff92dabef3714f489b6d4e84: done: 0.060769 sec","job_status":"done","queue_duration_s":1.989381,"scheduling_latency_s":1.98941,"redis_calls":10,"redis_duration_s":0.002256,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001329,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060769,"completed_at":"2026-09-03T10:16:05.060Z","load_balancing_strategy":"primary","db_duration_s":0.002033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577gitlab # [ 185.113239] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d95506dff13c19f2c8a11153","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d95506dff13c19f2c8a11153: start","job_status":"start","queue_duration_s":2.091499,"scheduling_latency_s":2.09154}1578gitlab # [ 185.128569] runuser[2209]: pam_unix(runuser:session): session closed for user gitlab1579gitlab # [ 185.130126] setup-gitlab-start[2191]: + systemctl restart gitlab-sidekiq.service1580gitlab # [ 185.144270] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d95506dff13c19f2c8a11153","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d95506dff13c19f2c8a11153: done: 0.031174 sec","job_status":"done","queue_duration_s":2.091499,"scheduling_latency_s":2.09154,"redis_calls":10,"redis_duration_s":0.0037300000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002086,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031174,"completed_at":"2026-09-03T10:16:05.132Z","load_balancing_strategy":"primary","db_duration_s":0.003358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1581gitlab # [ 185.145904] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7ea3e870abc0a42bbde5339d","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7ea3e870abc0a42bbde5339d: start","job_status":"start","queue_duration_s":2.124405,"scheduling_latency_s":2.124429}1582gitlab # [ 185.150934] systemd[1]: Stopping gitlab-sidekiq.service...1583gitlab # [ 185.151884] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.139Z","message":"Shutting down"}1584gitlab # [ 185.152741] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.140Z","message":"Scheduler exiting..."}1585gitlab # [ 185.153819] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.142Z","message":"Terminating quiet threads for default capsule"}1586gitlab # [ 185.155127] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.143Z","message":"Scheduler exiting..."}1587gitlab # [ 185.164584] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7ea3e870abc0a42bbde5339d","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7ea3e870abc0a42bbde5339d: done: 0.018725 sec","job_status":"done","queue_duration_s":2.124405,"scheduling_latency_s":2.124429,"redis_calls":10,"redis_duration_s":0.002645,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001747,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018725,"completed_at":"2026-09-03T10:16:05.153Z","load_balancing_strategy":"primary","db_duration_s":0.001431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588gitlab # [ 185.659056] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.646Z","message":"Bye!"}1589gitlab # [ 185.682432] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-09-03T10:16:05.671Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1622,"worker_id":"sidekiq_0","memwd_rss_bytes":947396608,"retry":0}1590gitlab # [ 187.048395] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1591gitlab # [ 187.050379] systemd[1]: Stopped gitlab-sidekiq.service.1592gitlab # [ 187.052235] systemd[1]: gitlab-sidekiq.service: Consumed 56.200s CPU time over 1min 53.085s wall clock time, 960.8M memory peak, 4.4M read from disk.1593gitlab # [ 187.059155] systemd[1]: Started gitlab-sidekiq.service.1594gitlab # [ 187.061741] setup-gitlab-start[2191]: + mkdir -p /var/lib/secrets1595gitlab # [ 187.063241] setup-gitlab-start[2191]: + echo nixbot-test-token-of-20-chars1596gitlab # [ 187.063900] setup-gitlab-start[2191]: + chmod 644 /var/lib/secrets/gitlab-token1597gitlab # [ 187.066304] setup-gitlab-start[2191]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1598gitlab # [ 187.942073] gitaly[920]: time="2026-09-03T10:16:07.930Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:17.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:07.930" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1599gitlab # [ 187.943592] gitaly[920]: time="2026-09-03T10:16:07.932Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:17.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:07.930" grpc.time_ms=1.551 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1600gitlab # [ 188.095991] gitaly[920]: time="2026-09-03T10:16:08.084Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:18.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.084" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1601gitlab # [ 188.118929] gitaly[920]: time="2026-09-03T10:16:08.107Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:18.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.084" grpc.time_ms=0.379 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1602gitlab # [ 188.195175] gitaly[920]: time="2026-09-03T10:16:08.183Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY diskcache=08e4d559-9c32-40d6-a58b-0e587916cfdb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-03T10:16:18.058" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.138" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603gitlab # [ 188.217599] gitaly[920]: time="2026-09-03T10:16:08.206Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1571 command.oublock=32 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-03T10:16:18.058" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.138" grpc.time_ms=45.448 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1604gitlab # [ 188.239339] gitaly[920]: time="2026-09-03T10:16:08.228Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:18.126" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.227" pid=920 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1605gitlab # [ 188.240470] gitaly[920]: time="2026-09-03T10:16:08.228Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:18.126" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.227" grpc.time_ms=0.428 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1606gitlab # [ 188.274267] gitaly[920]: time="2026-09-03T10:16:08.262Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:16:18.172" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.253" grpc.time_ms=8.492 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1607gitlab # [ 188.374182] gitaly[920]: time="2026-09-03T10:16:08.363Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:18.282" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.362" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1608gitlab # [ 188.396629] gitaly[920]: time="2026-09-03T10:16:08.385Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:18.282" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.362" grpc.time_ms=0.586 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1609gitlab # [ 188.473837] gitaly[920]: time="2026-09-03T10:16:08.462Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY diskcache=7ca21497-a399-4c74-b414-d7f187266e91 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-03T10:16:18.009" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.410" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1610gitlab # [ 188.475088] gitaly[920]: time="2026-09-03T10:16:08.462Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1580 command.oublock=32 command.real_time_ms=26 command.spawn_token_fork_ms=1 command.system_time_ms=16 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-03T10:16:18.009" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.410" grpc.time_ms=51.925 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1611gitlab # [ 188.503982] gitaly[920]: time="2026-09-03T10:16:08.492Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:18.092" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.492" pid=920 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1612gitlab # [ 188.505215] gitaly[920]: time="2026-09-03T10:16:08.493Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:18.092" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.492" grpc.time_ms=0.382 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1613gitlab # [ 188.536962] gitaly[920]: time="2026-09-03T10:16:08.525Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:16:18.116" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.516" grpc.time_ms=8.977 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1614gitlab # [ 188.568747] gitaly[920]: time="2026-09-03T10:16:08.557Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:16:18.157" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.557" grpc.time_ms=0.117 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1615gitlab # [ 188.579527] gitaly[920]: time="2026-09-03T10:16:08.568Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-03T10:16:18.167" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.568" grpc.time_ms=0.168 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1616gitlab # [ 189.014848] gitaly[920]: time="2026-09-03T10:16:09.003Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9HN0FXRW6W95WN87MVGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:16:18.072" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:08.993" grpc.time_ms=9.836 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1617gitlab # [ 189.373696] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:09 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 22891618gitlab # [ 189.378718] setup-gitlab-start[2191]: + mkdir -p /tmp/test-flake1619gitlab # [ 189.381385] setup-gitlab-start[2191]: + cd /tmp/test-flake1620gitlab # [ 189.383213] setup-gitlab-start[2191]: + git init -b master1621gitlab # [ 189.395830] setup-gitlab-start[2495]: Initialized empty Git repository in /tmp/test-flake/.git/1622gitlab # [ 189.398772] setup-gitlab-start[2191]: + git config user.name test1623gitlab # [ 189.400801] setup-gitlab-start[2191]: + git config user.email test@example.com1624gitlab # [ 189.403171] setup-gitlab-start[2191]: + cat1625gitlab # [ 189.404686] setup-gitlab-start[2191]: + git add flake.nix1626gitlab # [ 189.406590] setup-gitlab-start[2191]: + git commit -m 'initial commit'1627gitlab # [ 189.423695] setup-gitlab-start[2500]: [master (root-commit) eedce16] initial commit1628gitlab # [ 189.426164] setup-gitlab-start[2500]: 1 file changed, 10 insertions(+)1629gitlab # [ 189.426843] setup-gitlab-start[2500]: create mode 100644 flake.nix1630gitlab # [ 189.428025] setup-gitlab-start[2191]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1631gitlab # [ 189.433784] setup-gitlab-start[2191]: + git push -u origin master1632gitlab # [ 189.534101] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:09 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 801633gitlab # [ 189.725786] gitaly[920]: time="2026-09-03T10:16:09.713Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M1SWAXYC03H2MQ6NCM4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:09.700" grpc.time_ms=12.773 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1634gitlab # [ 189.754929] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:09 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1851635gitlab # [ 189.973481] gitaly[920]: time="2026-09-03T10:16:09.961Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:16:19.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:09.950" grpc.time_ms=11.138 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1636gitlab # [ 190.013978] gitaly[920]: time="2026-09-03T10:16:10.002Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-03T10:16:39.095" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-03T10:16:09.995" grpc.time_ms=7.172 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1637gitlab # [ 190.041244] gitaly[920]: time="2026-09-03T10:16:10.026Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 duration_ms=146ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/allowed"1638gitlab # [ 190.058053] gitaly[920]: time="2026-09-03T10:16:10.047Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 duration_ms=19ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/pre_receive"1639gitlab # [ 190.059542] gitaly[920]: time="2026-09-03T10:16:10.048Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:09.879" grpc.time_ms=169.076 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1640gitlab # [ 190.088247] sidekiq-cluster[2447]: {"severity":"INFO","time":"2026-09-03T10:16:10.060Z","message":"Starting cluster with 1 processes"}1641gitlab # [ 190.098704] gitaly[920]: time="2026-09-03T10:16:10.087Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:10.087" grpc.time_ms=0.437 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1642gitlab # [ 190.128219] gitaly[920]: time="2026-09-03T10:16:10.117Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:10.116" grpc.time_ms=0.377 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1643gitlab # [ 190.154962] gitaly[920]: time="2026-09-03T10:16:10.143Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:10.143" grpc.time_ms=0.3 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1644gitlab # [ 190.252417] gitaly[920]: time="2026-09-03T10:16:10.241Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:16:20.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:10.235" grpc.time_ms=5.784 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1645gitlab # [ 190.270399] gitaly[920]: time="2026-09-03T10:16:10.259Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1564 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-03T10:16:20.017" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:16:10.247" grpc.time_ms=11.245 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1646gitlab # [ 190.278184] gitaly[920]: time="2026-09-03T10:16:10.267Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 duration_ms=95ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/post_receive"1647gitlab # [ 190.279183] gitaly[920]: time="2026-09-03T10:16:10.267Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:10.171" grpc.time_ms=96.183 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1648gitlab # [ 190.302600] gitaly[920]: time="2026-09-03T10:16:10.291Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 diskcache=d36a7b0c-706c-4ec1-9a6e-02c24fd824f4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:09.820" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1649gitlab # [ 190.303187] gitaly[920]: time="2026-09-03T10:16:10.291Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=152 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=876544 command.minflt=13654 command.oublock=32 command.real_time_ms=463 command.spawn_token_fork_ms=0 command.system_time_ms=103 command.user_time_ms=49 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:09.820" grpc.time_ms=471.086 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1650gitlab # [ 190.303947] gitlab-workhorse[826]: time="2026-09-03T10:16:10Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521651gitlab # [ 190.304618] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:10 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 5391652gitlab # [ 190.305759] setup-gitlab-start[2504]: To http://gitlab/root/test-flake.git1653gitlab # [ 190.306163] setup-gitlab-start[2504]: * [new branch] master -> master1654gitlab # [ 190.330775] setup-gitlab-start[2504]: branch 'master' set up to track 'origin/master'.1655gitlab # [ 190.333851] systemd[1]: Finished setup-gitlab.service.1656gitlab # [ 190.336621] systemd[1]: Starting nixbot CI service...1657gitlab: (finished: waiting for unit setup-gitlab.service, in 191.24 seconds)1658gitlab: waiting for unit nixbot.service1659gitlab # [ 191.590092] nixbot[2597]: {"ts": "2026-09-03T10:16:11.578Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1660gitlab # [ 191.622818] nixbot[2597]: {"ts": "2026-09-03T10:16:11.611Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1661gitlab # [ 191.637867] nixbot[2597]: {"ts": "2026-09-03T10:16:11.626Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1662gitlab # [ 191.643652] nixbot[2597]: {"ts": "2026-09-03T10:16:11.632Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1663gitlab # [ 191.649113] nixbot[2597]: {"ts": "2026-09-03T10:16:11.637Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1664gitlab # [ 191.655784] nixbot[2597]: {"ts": "2026-09-03T10:16:11.644Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1665gitlab # [ 191.660832] nixbot[2597]: {"ts": "2026-09-03T10:16:11.649Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1666gitlab # [ 191.664877] nixbot[2597]: {"ts": "2026-09-03T10:16:11.653Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1667gitlab # [ 191.670987] nixbot[2597]: {"ts": "2026-09-03T10:16:11.659Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1668gitlab # [ 191.675286] nixbot[2597]: {"ts": "2026-09-03T10:16:11.664Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1669gitlab # [ 191.682898] nixbot[2597]: {"ts": "2026-09-03T10:16:11.671Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1670gitlab # [ 191.689477] nixbot[2597]: {"ts": "2026-09-03T10:16:11.678Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1671gitlab # [ 191.696097] nixbot[2597]: {"ts": "2026-09-03T10:16:11.684Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1672gitlab # [ 191.701630] nixbot[2597]: {"ts": "2026-09-03T10:16:11.690Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1673gitlab # [ 191.706970] nixbot[2597]: {"ts": "2026-09-03T10:16:11.695Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1674gitlab # [ 191.711550] nixbot[2597]: {"ts": "2026-09-03T10:16:11.700Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1675gitlab # [ 191.715926] nixbot[2597]: {"ts": "2026-09-03T10:16:11.704Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1676gitlab # [ 191.720182] nixbot[2597]: {"ts": "2026-09-03T10:16:11.708Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1677gitlab # [ 191.726319] nixbot[2597]: {"ts": "2026-09-03T10:16:11.715Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1678gitlab # [ 191.730447] nixbot[2597]: {"ts": "2026-09-03T10:16:11.719Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1679gitlab # [ 191.734579] nixbot[2597]: {"ts": "2026-09-03T10:16:11.723Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1680gitlab # [ 191.740717] nixbot[2597]: {"ts": "2026-09-03T10:16:11.729Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1681gitlab # [ 191.744898] nixbot[2597]: {"ts": "2026-09-03T10:16:11.733Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1682gitlab # [ 191.749398] nixbot[2597]: {"ts": "2026-09-03T10:16:11.738Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1683gitlab # [ 191.753396] nixbot[2597]: {"ts": "2026-09-03T10:16:11.742Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1684gitlab # [ 191.757894] nixbot[2597]: {"ts": "2026-09-03T10:16:11.746Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1685gitlab # [ 191.762251] nixbot[2597]: {"ts": "2026-09-03T10:16:11.751Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1686gitlab # [ 191.766420] nixbot[2597]: {"ts": "2026-09-03T10:16:11.755Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1687gitlab # [ 192.065339] nixbot[2597]: INFO: Started server process [2597]1688gitlab # [ 192.067448] nixbot[2597]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1689gitlab # [ 192.082496] nixbot[2597]: INFO: ::ffff:127.0.0.1:53232 - "GET /health HTTP/1.1" 200 OK1690gitlab # [ 192.088247] systemd[1]: Started nixbot CI service.1691gitlab # [ 192.090104] systemd[1]: Reached target Multi-User System.1692gitlab # [ 192.090820] systemd[1]: Startup finished in 1.133s (kernel) + 3.594s (initrd) + 3min 7.361s (userspace) = 3min 12.089s.1693gitlab # [ 192.291574] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2091694gitlab # [ 192.293223] nixbot[2597]: {"ts": "2026-09-03T10:16:12.281Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1695gitlab # [ 192.308705] nixbot[2597]: {"ts": "2026-09-03T10:16:12.297Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1696gitlab # [ 192.361547] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 411697gitlab # [ 192.364475] nixbot[2597]: {"ts": "2026-09-03T10:16:12.353Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1698gitlab # [ 192.366734] nixbot[2597]: {"ts": "2026-09-03T10:16:12.355Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1699gitlab # [ 192.482222] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 919 "" "python-httpx/0.28.1" 1091700gitlab # [ 192.484208] nixbot[2597]: {"ts": "2026-09-03T10:16:12.469Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1701gitlab # [ 192.595609] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 951702gitlab # [ 192.597466] nixbot[2597]: {"ts": "2026-09-03T10:16:12.585Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1703gitlab # [ 192.649817] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "python-httpx/0.28.1" 341704gitlab # [ 192.650817] nixbot[2597]: {"ts": "2026-09-03T10:16:12.639Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1705gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds)1706gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1707gitlab # [ 192.708073] nixbot[2597]: INFO: ::ffff:127.0.0.1:53248 - "GET /health HTTP/1.1" 200 OK1708gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds)1709subtest: project discovered and webhook registered1710gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1711gitlab # [ 192.767847] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "curl/8.21.0" 361712gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.06 seconds)1713(finished: subtest: project discovered and webhook registered, in 0.06 seconds)1714subtest: push delivers the webhook1715gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1716gitlab # [ 192.792280] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 919 "" "python-httpx/0.28.1" 1361717gitlab # [ 192.795289] nixbot[2597]: {"ts": "2026-09-03T10:16:12.783Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1718gitlab # [ 192.848840] gitaly[920]: time="2026-09-03T10:16:12.837Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9Q7K72F6XRMMFKY5K0M7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-03T10:16:42.024" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:16:12.824" grpc.time_ms=12.119 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1719gitlab # [ 192.854328] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 301720gitlab # [ 192.887092] gitaly[920]: time="2026-09-03T10:16:12.876Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9Q7K72F6XRMMFKY5K0M7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:16:42.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:16:12.875" grpc.time_ms=0.448 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1721gitlab # [ 192.907671] gitaly[920]: time="2026-09-03T10:16:12.896Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9Q7K72F6XRMMFKY5K0M7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-03T10:16:22.009" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:16:12.889" grpc.time_ms=6.849 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1722gitlab # [ 192.937087] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1371723gitlab # [ 192.938617] nixbot[2597]: {"ts": "2026-09-03T10:16:12.926Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1724gitlab # [ 192.943130] gitaly[920]: time="2026-09-03T10:16:12.932Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9Q9CTRM7KQ0MBEAPZ0KH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:12.922" grpc.time_ms=9.088 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1725gitlab # [ 192.943959] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 200 "" "git/2.55.0" 881726gitlab # Enumerating objects: 5, done.1727gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1728gitlab # Delta compression using up to 4 threads1729gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1730gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done.1731gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1732gitlab # [ 193.003972] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 611733gitlab # [ 193.007211] nixbot[2597]: {"ts": "2026-09-03T10:16:12.996Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1734gitlab # [ 193.012431] nixbot[2597]: {"ts": "2026-09-03T10:16:13.001Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1735gitlab # [ 193.076965] gitaly[920]: time="2026-09-03T10:16:13.065Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QF4XPXK231MTSCH89SG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.060" grpc.time_ms=5.45 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1736gitlab # [ 193.077972] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 381737gitlab # [ 193.129881] gitaly[920]: time="2026-09-03T10:16:13.118Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-03T10:16:43.011" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-03T10:16:13.111" grpc.time_ms=6.891 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1738gitlab # [ 193.143067] gitaly[920]: time="2026-09-03T10:16:13.131Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1KC9QD2972HN4KTYCB33A5W duration_ms=68ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/allowed"1739gitlab # [ 193.149761] gitaly[920]: time="2026-09-03T10:16:13.138Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QGCN5P1S2KA98QPZVWC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.132" pid=920 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1740gitlab # [ 193.159839] gitaly[920]: time="2026-09-03T10:16:13.138Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QGCN5P1S2KA98QPZVWC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.132" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1741gitlab # [ 193.160638] gitaly[920]: time="2026-09-03T10:16:13.148Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QGCN5P1S2KA98QPZVWC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.132" grpc.time_ms=16.243 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1742gitlab # [ 193.184645] gitlab-workhorse[826]: time="2026-09-03T10:16:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131743gitlab # [ 193.185463] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1051744gitlab # [ 193.194180] gitaly[920]: time="2026-09-03T10:16:13.183Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1KC9QD2972HN4KTYCB33A5W duration_ms=51ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/pre_receive"1745gitlab # [ 193.197079] gitaly[920]: time="2026-09-03T10:16:13.186Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:13.062" grpc.time_ms=123.664 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1746gitlab # [ 193.229475] gitaly[920]: time="2026-09-03T10:16:13.218Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:13.217" grpc.time_ms=0.617 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1747gitlab # [ 193.245384] gitaly[920]: time="2026-09-03T10:16:13.234Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QKW7ZN6SDM53GWKYSDR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.222" pid=920 remote_ip="2001:db8:1::1" request_sha=e0585bcc937da65569a488b3f87bb215173b1a6f response_bytes=224 span.kind=server system=grpc1748gitlab # [ 193.246077] gitaly[920]: time="2026-09-03T10:16:13.234Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1200 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QKW7ZN6SDM53GWKYSDR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.222" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1749gitlab # [ 193.246683] gitaly[920]: time="2026-09-03T10:16:13.234Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1200 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QKW7ZN6SDM53GWKYSDR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.222" grpc.time_ms=12.029 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1750gitlab # [ 193.279890] gitlab-workhorse[826]: time="2026-09-03T10:16:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2241751gitlab # [ 193.280621] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 224 "" "git/2.55.0" 881752gitlab # [ 193.304977] gitaly[920]: time="2026-09-03T10:16:13.291Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:13.290" grpc.time_ms=0.474 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1753gitlab # [ 193.329877] gitaly[920]: time="2026-09-03T10:16:13.318Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:13.318" grpc.time_ms=0.32 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1754gitlab # [ 193.380977] gitaly[920]: time="2026-09-03T10:16:13.369Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1KC9QD2972HN4KTYCB33A5W duration_ms=27ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/post_receive"1755gitlab # [ 193.383965] gitaly[920]: time="2026-09-03T10:16:13.372Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:13.342" grpc.time_ms=30.408 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1756gitlab # [ 193.403156] gitaly[920]: time="2026-09-03T10:16:13.392Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W diskcache=d9d5ddd1-fb70-4ae2-93eb-393619ebf23f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.016" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1757gitlab # [ 193.412540] gitaly[920]: time="2026-09-03T10:16:13.392Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=136 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13324 command.oublock=32 command.real_time_ms=366 command.spawn_token_fork_ms=0 command.system_time_ms=80 command.user_time_ms=56 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.016" grpc.time_ms=375.285 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1758gitlab # [ 193.426252] gitlab-workhorse[826]: time="2026-09-03T10:16:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521759gitlab # [ 193.427049] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:13 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 4521760gitlab # To http://gitlab/root/test-flake.git1761gitlab # eedce16..eba459b master -> master1762gitlab # [ 193.432480] gitaly[920]: time="2026-09-03T10:16:13.421Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QQPXYFQBSY5RM0Z3M4T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.413" grpc.time_ms=7.478 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1763gitlab # [ 193.433469] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1191764gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.68 seconds)1765gitlab: must succeed: git -C /tmp/test-flake rev-parse master1766gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1767gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1768gitlab # [ 193.477758] gitaly[920]: time="2026-09-03T10:16:13.466Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QVF04D0QMCS6JNGFCQ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.460" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1769gitlab # [ 193.478305] gitaly[920]: time="2026-09-03T10:16:13.466Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QVF04D0QMCS6JNGFCQ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.460" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1770gitlab # [ 193.492441] gitaly[920]: time="2026-09-03T10:16:13.481Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QVF04D0QMCS6JNGFCQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.460" grpc.time_ms=20.737 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1771gitlab # [ 193.512712] gitlab-workhorse[826]: time="2026-09-03T10:16:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671772gitlab # [ 193.513315] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 771773gitlab # [ 193.519818] nixbot[2597]: INFO: ::ffff:127.0.0.1:54734 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1774gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.06 seconds)1775gitlab # [ 193.558103] gitaly[920]: time="2026-09-03T10:16:13.547Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QY275Y633GR3YW0EA6M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.537" pid=920 remote_ip="2001:db8:1::1" request_sha=8c5f0baa3c8a777b4186bbf1ceb4fa03d6290583 response_bytes=336 span.kind=server system=grpc1776gitlab # [ 193.558894] gitaly[920]: time="2026-09-03T10:16:13.547Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QY275Y633GR3YW0EA6M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.537" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1777gitlab # [ 193.559479] gitaly[920]: time="2026-09-03T10:16:13.547Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QY275Y633GR3YW0EA6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.537" grpc.time_ms=9.794 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1778gitlab # [ 193.593970] gitlab-workhorse[826]: time="2026-09-03T10:16:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3361779gitlab # [ 193.594739] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 761780gitlab # [ 193.725260] gitaly[920]: time="2026-09-03T10:16:13.714Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9R1MKT1PQHF1GMVNNE3W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.701" grpc.time_ms=12.189 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1781gitlab # [ 193.727335] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 951782gitlab # [ 193.807120] gitaly[920]: time="2026-09-03T10:16:13.795Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9R5KX6Q00KX8PZH6P8PK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.783" pid=920 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1783gitlab # [ 193.810729] gitaly[920]: time="2026-09-03T10:16:13.796Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1176 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9R5KX6Q00KX8PZH6P8PK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.783" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1784gitlab # [ 193.813253] gitlab-workhorse[826]: time="2026-09-03T10:16:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091785gitlab # [ 193.814344] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 501786gitlab # [ 193.815545] gitaly[920]: time="2026-09-03T10:16:13.796Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1176 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9R5KX6Q00KX8PZH6P8PK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:13.783" grpc.time_ms=12.202 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1787gitlab # [ 193.964862] gitaly[920]: time="2026-09-03T10:16:13.953Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:16:43.051" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:16:13.952" grpc.time_ms=0.742 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1788gitlab # [ 194.033783] gitaly[920]: time="2026-09-03T10:16:14.022Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-03T10:16:44.013" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:16:14.013" grpc.time_ms=9.055 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789gitlab # [ 194.085561] gitaly[920]: time="2026-09-03T10:16:14.074Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:16:24.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:16:14.065" grpc.time_ms=8.669 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790gitlab # [ 194.128044] gitaly[920]: time="2026-09-03T10:16:14.116Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:16:24.049" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:16:14.109" grpc.time_ms=7.088 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1791gitlab # [ 194.222160] gitaly[920]: time="2026-09-03T10:16:14.210Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-03T10:16:24.149" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:14.210" grpc.time_ms=0.344 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1792gitlab # [ 194.267901] gitaly[920]: time="2026-09-03T10:16:14.256Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-03T10:16:24.184" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:16:14.244" grpc.time_ms=11.708 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1793gitlab # [ 194.348307] gitaly[920]: time="2026-09-03T10:16:14.336Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:14.336" grpc.time_ms=0.57 pid=920 span.kind=server system=grpc1794gitlab # [ 194.389946] gitaly[920]: time="2026-09-03T10:16:14.378Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:16:14.378" grpc.time_ms=0.434 pid=920 span.kind=server system=grpc1795gitlab # [ 194.413171] gitaly[920]: time="2026-09-03T10:16:14.401Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X diskcache=7500615d-1a3d-4f34-8004-2ed6c2ae92ca grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:16:24.226" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:14.286" pid=920 ref=refs/keep-around/eedce16a998e57e3e500c437ccc385d7d5ac4a49 remote_ip="2001:db8:1::1" revision=eedce16a998e57e3e500c437ccc385d7d5ac4a49 span.kind=server system=grpc username=root1796gitlab # [ 194.414370] gitaly[920]: time="2026-09-03T10:16:14.402Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=77 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5355 command.oublock=8 command.real_time_ms=106 command.spawn_token_fork_ms=0 command.system_time_ms=48 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:16:24.226" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:16:14.286" grpc.time_ms=115.499 pid=920 ref=refs/keep-around/eedce16a998e57e3e500c437ccc385d7d5ac4a49 remote_ip="2001:db8:1::1" revision=eedce16a998e57e3e500c437ccc385d7d5ac4a49 span.kind=server system=grpc username=root1797gitlab # [ 194.440313] gitaly[920]: time="2026-09-03T10:16:14.429Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:16:44.428" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:16:14.428" grpc.time_ms=0.471 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1799gitlab # [ 194.554958] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 6471800gitlab # [ 194.557724] nixbot[2597]: {"ts": "2026-09-03T10:16:14.545Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 \"HTTP/1.1 201 Created\""}1801gitlab # [ 194.566100] nixbot[2597]: INFO: ::ffff:127.0.0.1:54736 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1802gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.05 seconds)1803gitlab # [ 194.578953] nixbot[2597]: {"ts": "2026-09-03T10:16:14.567Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-eedce16a998e-0"}1804gitlab # [ 194.746474] nixbot[2597]: {"ts": "2026-09-03T10:16:14.735Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-eedce16a998e-0", "argv0": "bwrap"}1805gitlab # [ 194.865170] systemd[1]: Started Nix Daemon.1806gitlab # [ 194.931492] nix-daemon[2825]: accepted connection from pid 2820, user nixbot1807gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1808gitlab # [ 195.605835] nixbot[2597]: INFO: ::ffff:127.0.0.1:54750 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1809gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1810gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1811gitlab # [ 196.645546] nixbot[2597]: INFO: ::ffff:127.0.0.1:54764 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1812gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1813gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1814gitlab # [ 197.684760] nixbot[2597]: INFO: ::ffff:127.0.0.1:54772 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1815gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1816gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1817gitlab # [ 198.724199] nixbot[2597]: INFO: ::ffff:127.0.0.1:54780 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1818gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1819gitlab # [ 199.299169] nix-daemon[2825]: accepted connection from pid 2852, user nixbot1820gitlab # [ 199.316650] nix-daemon[2825]: accepted connection from pid 2820, user nixbot1821gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1822gitlab # [ 199.763540] nixbot[2597]: INFO: ::ffff:127.0.0.1:54786 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1823gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1824gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1825gitlab # [ 200.800929] nixbot[2597]: INFO: ::ffff:127.0.0.1:54792 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1826gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1827gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1828gitlab # [ 201.836872] nixbot[2597]: INFO: ::ffff:127.0.0.1:54794 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1829gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1830gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1831gitlab # [ 202.873794] nixbot[2597]: INFO: ::ffff:127.0.0.1:54808 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1832gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1833gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1834gitlab # [ 203.910062] nixbot[2597]: INFO: ::ffff:127.0.0.1:55770 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1835gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1836gitlab # [ 204.057787] nix-daemon[2825]: accepted connection from pid 2820, user nixbot1837gitlab # [ 204.147202] nix-daemon[2825]: accepted connection from pid 2884, user nixbot1838gitlab # [ 204.169997] gitaly[920]: time="2026-09-03T10:16:24.158Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCA29RBQKG4KM81YPHQJ00 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:16:54.155" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:16:24.156" grpc.time_ms=0.839 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 204.255502] gitaly[920]: time="2026-09-03T10:16:24.244Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KCA29RBQKG4KM81YPHQJ00 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:16:34.174" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:16:24.235" grpc.time_ms=8.875 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840gitlab # [ 204.317859] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1851841gitlab # [ 204.324648] nixbot[2597]: {"ts": "2026-09-03T10:16:24.313Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 \"HTTP/1.1 201 Created\""}1842gitlab # [ 204.366437] gitaly[920]: time="2026-09-03T10:16:24.355Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCA2G2FPZW359GYPNEVBAC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:16:54.353" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:16:24.354" grpc.time_ms=0.657 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1843gitlab # [ 204.433183] gitaly[920]: time="2026-09-03T10:16:24.421Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KCA2G2FPZW359GYPNEVBAC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:16:34.181" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:16:24.411" grpc.time_ms=10.06 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1844gitlab # [ 204.490519] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1561845gitlab # [ 204.492147] nixbot[2597]: {"ts": "2026-09-03T10:16:24.480Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 \"HTTP/1.1 201 Created\""}1846gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1847gitlab # [ 204.948379] nixbot[2597]: INFO: ::ffff:127.0.0.1:55784 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1848gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1849gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1850gitlab # [ 205.985167] nixbot[2597]: INFO: ::ffff:127.0.0.1:55792 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1851gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1852gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1853gitlab # [ 207.020842] nixbot[2597]: INFO: ::ffff:127.0.0.1:55796 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1854gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1855gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1856gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1857gitlab # [ 208.056662] nixbot[2597]: INFO: ::ffff:127.0.0.1:55806 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1858gitlab # [ 208.499076] nix-daemon[2825]: accepted connection from pid 2922, user nixbot1859gitlab # [ 208.572786] gitaly[920]: time="2026-09-03T10:16:28.560Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCA6K3TH9Q4EVSPB96E31T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:16:58.459" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:16:28.559" grpc.time_ms=0.852 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1860gitlab # [ 208.657735] gitaly[920]: time="2026-09-03T10:16:28.646Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCA6K3TH9Q4EVSPB96E31T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:16:38.236" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:16:28.637" grpc.time_ms=9.046 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1861gitlab # [ 208.728603] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2001862gitlab # [ 208.734754] nixbot[2597]: {"ts": "2026-09-03T10:16:28.717Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 \"HTTP/1.1 201 Created\""}1863gitlab # [ 208.894104] gitaly[920]: time="2026-09-03T10:16:28.882Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KCA6X61TSATFZ4CMWY6ZWP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:28.870" grpc.time_ms=12.404 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1864gitlab # [ 208.896425] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 441865gitlab # [ 208.973186] gitaly[920]: time="2026-09-03T10:16:28.962Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCA6ZMKBD13VABGJ17F58Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:28.950" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1866gitlab # [ 208.975207] gitaly[920]: time="2026-09-03T10:16:28.962Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCA6ZMKBD13VABGJ17F58Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:28.950" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1867gitlab # [ 208.975838] gitlab-workhorse[826]: time="2026-09-03T10:16:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671868gitlab # [ 208.978346] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:16:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 451869gitlab # [ 208.980573] gitaly[920]: time="2026-09-03T10:16:28.962Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCA6ZMKBD13VABGJ17F58Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:16:28.950" grpc.time_ms=12.214 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1870gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1871gitlab # [ 209.080347] nixbot[2597]: INFO: ::ffff:127.0.0.1:55812 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1872gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1873gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1874gitlab # [ 210.118783] nixbot[2597]: INFO: ::ffff:127.0.0.1:55822 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1875gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1876gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1877gitlab # [ 211.156814] nixbot[2597]: INFO: ::ffff:127.0.0.1:55836 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1878gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1879gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1880gitlab # [ 212.194288] nixbot[2597]: INFO: ::ffff:127.0.0.1:55844 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1881gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1882gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1883gitlab # [ 213.232498] nixbot[2597]: INFO: ::ffff:127.0.0.1:45238 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1884gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1885gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1886gitlab # [ 214.267485] nixbot[2597]: INFO: ::ffff:127.0.0.1:45242 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1887gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1888gitlab # [ 214.468247] sidekiq-cluster[2556]: 2026-09-03T10:16:34.456Z pid=2556 tid=xuc INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2556, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>434487296}1889gitlab # [ 214.636592] sidekiq-cluster[2556]: W, [2026-09-03T10:16:34.625191 #2556] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1890gitlab # [ 214.674907] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:34.663Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1891gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1892gitlab # [ 215.301894] nixbot[2597]: INFO: ::ffff:127.0.0.1:45244 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1893gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1894gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1895gitlab # [ 216.336742] nixbot[2597]: INFO: ::ffff:127.0.0.1:45256 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1896gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1897gitlab # [ 216.342581] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:36.330Z","message":"GitLab reliable fetch activated!"}1898gitlab # [ 216.346670] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:36.331Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1899gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1900gitlab # [ 217.377437] nixbot[2597]: INFO: ::ffff:127.0.0.1:45264 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1901gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1902gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1903gitlab # [ 218.411631] nixbot[2597]: INFO: ::ffff:127.0.0.1:45270 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1904gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1905gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1906gitlab # [ 219.445563] nixbot[2597]: INFO: ::ffff:127.0.0.1:45286 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1907gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1908gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1909gitlab # [ 220.479429] nixbot[2597]: INFO: ::ffff:127.0.0.1:45298 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1910gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1911gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1912gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1913gitlab # [ 221.512969] nixbot[2597]: INFO: ::ffff:127.0.0.1:45308 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1914gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1915gitlab # [ 222.545129] nixbot[2597]: INFO: ::ffff:127.0.0.1:45324 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1916gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1917gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1918gitlab # [ 223.579173] nixbot[2597]: INFO: ::ffff:127.0.0.1:44630 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1919gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1920gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1921gitlab # [ 224.613124] nixbot[2597]: INFO: ::ffff:127.0.0.1:44646 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1922gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1923gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1924gitlab # [ 225.647265] nixbot[2597]: INFO: ::ffff:127.0.0.1:44650 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1925gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1926gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1927gitlab # [ 226.682345] nixbot[2597]: INFO: ::ffff:127.0.0.1:44664 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1928gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1929gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1930gitlab # [ 227.716750] nixbot[2597]: INFO: ::ffff:127.0.0.1:44674 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1931gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1932gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1933gitlab # [ 228.750771] nixbot[2597]: INFO: ::ffff:127.0.0.1:44676 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1934gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1935gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1936gitlab # [ 229.785606] nixbot[2597]: INFO: ::ffff:127.0.0.1:44678 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1937gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1938gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1939gitlab # [ 230.820067] nixbot[2597]: INFO: ::ffff:127.0.0.1:44680 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1940gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1941gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1942gitlab # [ 231.854469] nixbot[2597]: INFO: ::ffff:127.0.0.1:44696 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1943gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1944gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1945gitlab # [ 232.889104] nixbot[2597]: INFO: ::ffff:127.0.0.1:44706 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1946gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1947gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1948gitlab # [ 233.916831] nixbot[2597]: INFO: ::ffff:127.0.0.1:47326 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1949gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1950gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1951gitlab # [ 234.954380] nixbot[2597]: INFO: ::ffff:127.0.0.1:47332 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1952gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1953gitlab # [ 235.377652] postgres[3058]: [3058] ERROR: relation "table_without_models" does not exist1954gitlab # [ 235.380476] postgres[3058]: [3058] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1955gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1956gitlab # [ 235.992501] nixbot[2597]: INFO: ::ffff:127.0.0.1:47346 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1957gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1958gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1959gitlab # [ 237.029744] nixbot[2597]: INFO: ::ffff:127.0.0.1:47352 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1960gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)1961gitlab # [ 237.747666] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:57.736Z","message":"Booted Rails 7.2.3.1 application in production environment"}1962gitlab # [ 237.751885] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:57.740Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1963gitlab # [ 237.756561] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:57.740Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1964gitlab # [ 237.760560] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:57.740Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1965gitlab # [ 237.765117] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:57.741Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1966gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1967gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.03 seconds)1968gitlab # [ 238.067665] nixbot[2597]: INFO: ::ffff:127.0.0.1:47362 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1969gitlab # [ 238.961097] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:58.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"dc6580260f0cc9f806e2aa6a","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-dc6580260f0cc9f806e2aa6a: start","job_status":"start","queue_duration_s":55.938952,"scheduling_latency_s":55.939018}1970gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'1971gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.06 seconds)1972gitlab # [ 239.068915] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"dc6580260f0cc9f806e2aa6a","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-dc6580260f0cc9f806e2aa6a: done: 0.108643 sec","job_status":"done","queue_duration_s":55.938952,"scheduling_latency_s":55.939018,"redis_calls":25,"redis_duration_s":0.005607,"redis_read_bytes":912,"redis_write_bytes":2573,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003255,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1631,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001388,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.108643,"completed_at":"2026-09-03T10:16:59.057Z","load_balancing_strategy":"primary","db_duration_s":0.016722,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973gitlab # [ 239.141806] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"99d33935c9def86b3695e773","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-99d33935c9def86b3695e773: start","job_status":"start","queue_duration_s":56.048525,"scheduling_latency_s":56.048543}1974gitlab # [ 239.152922] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"99d33935c9def86b3695e773","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-99d33935c9def86b3695e773: done: 0.026949 sec","job_status":"done","queue_duration_s":56.048525,"scheduling_latency_s":56.048543,"redis_calls":10,"redis_duration_s":0.004026,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001818,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002208,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026949,"completed_at":"2026-09-03T10:16:59.085Z","load_balancing_strategy":"primary","db_duration_s":0.002686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1975gitlab # [ 239.187699] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7325adaa31ce92e1738957f4","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7325adaa31ce92e1738957f4: start","job_status":"start","queue_duration_s":56.077679,"scheduling_latency_s":56.077713}1976gitlab # [ 239.199431] nixbot[2597]: INFO: ::ffff:127.0.0.1:47368 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK1977gitlab # [ 239.200418] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7325adaa31ce92e1738957f4","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7325adaa31ce92e1738957f4: done: 0.020333 sec","job_status":"done","queue_duration_s":56.077679,"scheduling_latency_s":56.077713,"redis_calls":10,"redis_duration_s":0.002667,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00116,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001507,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020333,"completed_at":"2026-09-03T10:16:59.108Z","load_balancing_strategy":"primary","db_duration_s":0.001642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978gitlab # [ 239.201903] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fa8a798de106a4109f72ebe8","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fa8a798de106a4109f72ebe8: start","job_status":"start","queue_duration_s":56.099088,"scheduling_latency_s":56.099103}1979gitlab # [ 239.207312] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fa8a798de106a4109f72ebe8","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fa8a798de106a4109f72ebe8: done: 0.026163 sec","job_status":"done","queue_duration_s":56.099088,"scheduling_latency_s":56.099103,"redis_calls":10,"redis_duration_s":0.003868,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001436,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002432,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026163,"completed_at":"2026-09-03T10:16:59.135Z","load_balancing_strategy":"primary","db_duration_s":0.002372,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1980gitlab # [ 239.255252] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"14b542ec177108bab6ff5ca8","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-14b542ec177108bab6ff5ca8: start","job_status":"start","queue_duration_s":56.127201,"scheduling_latency_s":56.12723}1981gitlab # [ 239.256211] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"14b542ec177108bab6ff5ca8","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-14b542ec177108bab6ff5ca8: done: 0.019592 sec","job_status":"done","queue_duration_s":56.127201,"scheduling_latency_s":56.12723,"redis_calls":10,"redis_duration_s":0.002583,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001218,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001365,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019592,"completed_at":"2026-09-03T10:16:59.156Z","load_balancing_strategy":"primary","db_duration_s":0.001544,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1982gitlab # [ 239.298312] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bce4e9ba4cbbe4cbfc0288ec","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bce4e9ba4cbbe4cbfc0288ec: start","job_status":"start","queue_duration_s":56.147761,"scheduling_latency_s":56.147774}1983gitlab # [ 239.300836] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bce4e9ba4cbbe4cbfc0288ec","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bce4e9ba4cbbe4cbfc0288ec: done: 0.011062 sec","job_status":"done","queue_duration_s":56.147761,"scheduling_latency_s":56.147774,"redis_calls":10,"redis_duration_s":0.00166,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011062,"completed_at":"2026-09-03T10:16:59.168Z","load_balancing_strategy":"primary","db_duration_s":0.00096,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1984gitlab # [ 239.363883] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8f36f5689a33fb7d3c74e749","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8f36f5689a33fb7d3c74e749: start","job_status":"start","queue_duration_s":56.159809,"scheduling_latency_s":56.159821}1985gitlab # [ 239.364808] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8f36f5689a33fb7d3c74e749","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8f36f5689a33fb7d3c74e749: done: 0.01081 sec","job_status":"done","queue_duration_s":56.159809,"scheduling_latency_s":56.159821,"redis_calls":10,"redis_duration_s":0.001679,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01081,"completed_at":"2026-09-03T10:16:59.180Z","load_balancing_strategy":"primary","db_duration_s":0.000945,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1986gitlab # [ 239.410982] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f18334dec8e5d9ec24c9c7c","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f18334dec8e5d9ec24c9c7c: start","job_status":"start","queue_duration_s":56.171573,"scheduling_latency_s":56.171587}1987gitlab # [ 239.411608] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f18334dec8e5d9ec24c9c7c","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f18334dec8e5d9ec24c9c7c: done: 0.01387 sec","job_status":"done","queue_duration_s":56.171573,"scheduling_latency_s":56.171587,"redis_calls":10,"redis_duration_s":0.001821,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000642,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001179,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01387,"completed_at":"2026-09-03T10:16:59.195Z","load_balancing_strategy":"primary","db_duration_s":0.001308,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1988gitlab # [ 239.459800] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a1cfd17425698f3873e9746a","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a1cfd17425698f3873e9746a: start","job_status":"start","queue_duration_s":56.186491,"scheduling_latency_s":56.186508}1989gitlab # [ 239.460811] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a1cfd17425698f3873e9746a","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a1cfd17425698f3873e9746a: done: 0.011486 sec","job_status":"done","queue_duration_s":56.186491,"scheduling_latency_s":56.186508,"redis_calls":10,"redis_duration_s":0.001271,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011486,"completed_at":"2026-09-03T10:16:59.208Z","load_balancing_strategy":"primary","db_duration_s":0.001101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1990gitlab # [ 239.519220] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"28a62c3a130bce4f586b4f11","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-28a62c3a130bce4f586b4f11: start","job_status":"start","queue_duration_s":56.198755,"scheduling_latency_s":56.198769}1991gitlab # [ 239.520136] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"28a62c3a130bce4f586b4f11","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-28a62c3a130bce4f586b4f11: done: 0.010424 sec","job_status":"done","queue_duration_s":56.198755,"scheduling_latency_s":56.198769,"redis_calls":10,"redis_duration_s":0.001245,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010424,"completed_at":"2026-09-03T10:16:59.219Z","load_balancing_strategy":"primary","db_duration_s":0.001053,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1992gitlab # [ 239.526980] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f49277079d332d65879a3209","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f49277079d332d65879a3209: start","job_status":"start","queue_duration_s":56.210004,"scheduling_latency_s":56.210017}1993gitlab # [ 239.538781] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f49277079d332d65879a3209","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f49277079d332d65879a3209: done: 0.010324 sec","job_status":"done","queue_duration_s":56.210004,"scheduling_latency_s":56.210017,"redis_calls":10,"redis_duration_s":0.0012079999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000717,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010324,"completed_at":"2026-09-03T10:16:59.230Z","load_balancing_strategy":"primary","db_duration_s":0.000903,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1994gitlab # [ 239.581342] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8b901d2fb7f2459d82e23ffb","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8b901d2fb7f2459d82e23ffb: start","job_status":"start","queue_duration_s":56.221145,"scheduling_latency_s":56.221158}1995gitlab # [ 239.585627] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8b901d2fb7f2459d82e23ffb","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8b901d2fb7f2459d82e23ffb: done: 0.010453 sec","job_status":"done","queue_duration_s":56.221145,"scheduling_latency_s":56.221158,"redis_calls":10,"redis_duration_s":0.00126,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010453,"completed_at":"2026-09-03T10:16:59.241Z","load_balancing_strategy":"primary","db_duration_s":0.000933,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1996gitlab # [ 239.629914] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e08897e8330ae9804f7c9788","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e08897e8330ae9804f7c9788: start","job_status":"start","queue_duration_s":56.232417,"scheduling_latency_s":56.232431}1997gitlab # [ 239.632580] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e08897e8330ae9804f7c9788","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e08897e8330ae9804f7c9788: done: 0.01053 sec","job_status":"done","queue_duration_s":56.232417,"scheduling_latency_s":56.232431,"redis_calls":10,"redis_duration_s":0.001226,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01053,"completed_at":"2026-09-03T10:16:59.253Z","load_balancing_strategy":"primary","db_duration_s":0.000941,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1998gitlab # [ 239.680671] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ba1b80826e7befbf7988cb04","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ba1b80826e7befbf7988cb04: start","job_status":"start","queue_duration_s":56.243748,"scheduling_latency_s":56.243761}1999gitlab # [ 239.681636] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ba1b80826e7befbf7988cb04","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ba1b80826e7befbf7988cb04: done: 0.010571 sec","job_status":"done","queue_duration_s":56.243748,"scheduling_latency_s":56.243761,"redis_calls":10,"redis_duration_s":0.001206,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000717,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010571,"completed_at":"2026-09-03T10:16:59.264Z","load_balancing_strategy":"primary","db_duration_s":0.000939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2000gitlab # [ 239.726902] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"81ae425979110367164be60e","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-81ae425979110367164be60e: start","job_status":"start","queue_duration_s":56.255132,"scheduling_latency_s":56.255145}2001gitlab # [ 239.727854] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"81ae425979110367164be60e","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-81ae425979110367164be60e: done: 0.010332 sec","job_status":"done","queue_duration_s":56.255132,"scheduling_latency_s":56.255145,"redis_calls":10,"redis_duration_s":0.001215,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000705,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010332,"completed_at":"2026-09-03T10:16:59.275Z","load_balancing_strategy":"primary","db_duration_s":0.000916,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2002gitlab # [ 239.764772] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3cf4688e35d9a1d970fe240d","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3cf4688e35d9a1d970fe240d: start","job_status":"start","queue_duration_s":56.266304,"scheduling_latency_s":56.266318}2003gitlab # [ 239.776101] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3cf4688e35d9a1d970fe240d","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3cf4688e35d9a1d970fe240d: done: 0.010348 sec","job_status":"done","queue_duration_s":56.266304,"scheduling_latency_s":56.266318,"redis_calls":10,"redis_duration_s":0.001197,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010348,"completed_at":"2026-09-03T10:16:59.286Z","load_balancing_strategy":"primary","db_duration_s":0.000929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab # [ 239.836265] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"54dac0aabe16396f785e4156","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-54dac0aabe16396f785e4156: start","job_status":"start","queue_duration_s":56.277543,"scheduling_latency_s":56.277557}2005gitlab # [ 239.836903] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"54dac0aabe16396f785e4156","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-54dac0aabe16396f785e4156: done: 0.010414 sec","job_status":"done","queue_duration_s":56.277543,"scheduling_latency_s":56.277557,"redis_calls":10,"redis_duration_s":0.00134,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010414,"completed_at":"2026-09-03T10:16:59.298Z","load_balancing_strategy":"primary","db_duration_s":0.000966,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2006gitlab # [ 239.857978] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"42524739f4ee30f7eab44b58","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-42524739f4ee30f7eab44b58: start","job_status":"start","queue_duration_s":56.288741,"scheduling_latency_s":56.288754}2007gitlab # [ 239.869670] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"42524739f4ee30f7eab44b58","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-42524739f4ee30f7eab44b58: done: 0.010265 sec","job_status":"done","queue_duration_s":56.288741,"scheduling_latency_s":56.288754,"redis_calls":10,"redis_duration_s":0.001241,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010265,"completed_at":"2026-09-03T10:16:59.309Z","load_balancing_strategy":"primary","db_duration_s":0.000929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 239.921062] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"37bd8ecab3b4a54795d0b088","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-37bd8ecab3b4a54795d0b088: start","job_status":"start","queue_duration_s":56.299802,"scheduling_latency_s":56.299815}2009gitlab # [ 239.921875] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"37bd8ecab3b4a54795d0b088","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-37bd8ecab3b4a54795d0b088: done: 0.011266 sec","job_status":"done","queue_duration_s":56.299802,"scheduling_latency_s":56.299815,"redis_calls":10,"redis_duration_s":0.001348,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000756,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011266,"completed_at":"2026-09-03T10:16:59.321Z","load_balancing_strategy":"primary","db_duration_s":0.001014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2010gitlab # [ 239.978685] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e2c8f5e922f32edfb49f867f","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e2c8f5e922f32edfb49f867f: start","job_status":"start","queue_duration_s":56.311854,"scheduling_latency_s":56.311867}2011gitlab # [ 239.979553] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e2c8f5e922f32edfb49f867f","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e2c8f5e922f32edfb49f867f: done: 0.010334 sec","job_status":"done","queue_duration_s":56.311854,"scheduling_latency_s":56.311867,"redis_calls":10,"redis_duration_s":0.001232,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010334,"completed_at":"2026-09-03T10:16:59.332Z","load_balancing_strategy":"primary","db_duration_s":0.000945,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2012gitlab # [ 240.023711] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"29fcb1fe38c849a6f3e5191f","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-29fcb1fe38c849a6f3e5191f: start","job_status":"start","queue_duration_s":56.323016,"scheduling_latency_s":56.32303}2013gitlab # [ 240.024570] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"29fcb1fe38c849a6f3e5191f","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-29fcb1fe38c849a6f3e5191f: done: 0.010461 sec","job_status":"done","queue_duration_s":56.323016,"scheduling_latency_s":56.32303,"redis_calls":10,"redis_duration_s":0.001252,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000745,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010461,"completed_at":"2026-09-03T10:16:59.343Z","load_balancing_strategy":"primary","db_duration_s":0.000965,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 240.078698] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e52874867d6e0dc83de5facc","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e52874867d6e0dc83de5facc: start","job_status":"start","queue_duration_s":56.334437,"scheduling_latency_s":56.33445}2015gitlab # [ 240.079653] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e52874867d6e0dc83de5facc","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e52874867d6e0dc83de5facc: done: 0.010399 sec","job_status":"done","queue_duration_s":56.334437,"scheduling_latency_s":56.33445,"redis_calls":10,"redis_duration_s":0.001266,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000746,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010399,"completed_at":"2026-09-03T10:16:59.355Z","load_balancing_strategy":"primary","db_duration_s":0.000983,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2016gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'2017gitlab # [ 240.122099] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b00ef263cf574311120e4462","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b00ef263cf574311120e4462: start","job_status":"start","queue_duration_s":56.345671,"scheduling_latency_s":56.345684}2018gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)2019gitlab # [ 240.122751] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b00ef263cf574311120e4462","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b00ef263cf574311120e4462: done: 0.01049 sec","job_status":"done","queue_duration_s":56.345671,"scheduling_latency_s":56.345684,"redis_calls":10,"redis_duration_s":0.001234,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01049,"completed_at":"2026-09-03T10:16:59.366Z","load_balancing_strategy":"primary","db_duration_s":0.00097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2020gitlab # [ 240.168696] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"401477c590a36e27ad186e28","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-401477c590a36e27ad186e28: start","job_status":"start","queue_duration_s":56.356991,"scheduling_latency_s":56.357005}2021gitlab # [ 240.169610] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"401477c590a36e27ad186e28","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-401477c590a36e27ad186e28: done: 0.010457 sec","job_status":"done","queue_duration_s":56.356991,"scheduling_latency_s":56.357005,"redis_calls":10,"redis_duration_s":0.001211,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000711,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010457,"completed_at":"2026-09-03T10:16:59.377Z","load_balancing_strategy":"primary","db_duration_s":0.000895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2022gitlab # [ 240.210503] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bde615ffb30ed762114f46e3","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bde615ffb30ed762114f46e3: start","job_status":"start","queue_duration_s":56.368268,"scheduling_latency_s":56.368281}2023gitlab # [ 240.210910] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bde615ffb30ed762114f46e3","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bde615ffb30ed762114f46e3: done: 0.010409 sec","job_status":"done","queue_duration_s":56.368268,"scheduling_latency_s":56.368281,"redis_calls":10,"redis_duration_s":0.0012980000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000813,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010409,"completed_at":"2026-09-03T10:16:59.388Z","load_balancing_strategy":"primary","db_duration_s":0.000921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024gitlab # [ 240.226130] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ccb7cc09e30e005862d254fc","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ccb7cc09e30e005862d254fc: start","job_status":"start","queue_duration_s":56.379485,"scheduling_latency_s":56.379497}2025gitlab # [ 240.267428] nixbot[2597]: INFO: ::ffff:127.0.0.1:47382 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK2026gitlab # [ 240.268133] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ccb7cc09e30e005862d254fc","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ccb7cc09e30e005862d254fc: done: 0.010249 sec","job_status":"done","queue_duration_s":56.379485,"scheduling_latency_s":56.379497,"redis_calls":10,"redis_duration_s":0.001239,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000727,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010249,"completed_at":"2026-09-03T10:16:59.399Z","load_balancing_strategy":"primary","db_duration_s":0.000941,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2027gitlab # [ 240.271130] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"07644e3f45a76575393a9995","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-07644e3f45a76575393a9995: start","job_status":"start","queue_duration_s":56.390569,"scheduling_latency_s":56.390581}2028gitlab # [ 240.281831] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"07644e3f45a76575393a9995","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-07644e3f45a76575393a9995: done: 0.0107 sec","job_status":"done","queue_duration_s":56.390569,"scheduling_latency_s":56.390581,"redis_calls":10,"redis_duration_s":0.0012040000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000698,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0107,"completed_at":"2026-09-03T10:16:59.411Z","load_balancing_strategy":"primary","db_duration_s":0.001238,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2029gitlab # [ 240.326739] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e09cd66afcda220247da3ec9","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e09cd66afcda220247da3ec9: start","job_status":"start","queue_duration_s":56.402087,"scheduling_latency_s":56.4021}2030gitlab # [ 240.327619] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e09cd66afcda220247da3ec9","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e09cd66afcda220247da3ec9: done: 0.010475 sec","job_status":"done","queue_duration_s":56.402087,"scheduling_latency_s":56.4021,"redis_calls":10,"redis_duration_s":0.001227,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000729,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010475,"completed_at":"2026-09-03T10:16:59.422Z","load_balancing_strategy":"primary","db_duration_s":0.000932,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2031gitlab # [ 240.358092] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"07f60a1c2ac79bd94dd0a15e","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-07f60a1c2ac79bd94dd0a15e: start","job_status":"start","queue_duration_s":56.413448,"scheduling_latency_s":56.413461}2032gitlab # [ 240.403442] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"07f60a1c2ac79bd94dd0a15e","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-07f60a1c2ac79bd94dd0a15e: done: 0.01049 sec","job_status":"done","queue_duration_s":56.413448,"scheduling_latency_s":56.413461,"redis_calls":10,"redis_duration_s":0.001228,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01049,"completed_at":"2026-09-03T10:16:59.434Z","load_balancing_strategy":"primary","db_duration_s":0.000907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2033gitlab # [ 240.404498] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"82318735e638ef2cf41a3c2d","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-82318735e638ef2cf41a3c2d: start","job_status":"start","queue_duration_s":56.424777,"scheduling_latency_s":56.424791}2034gitlab # [ 240.412258] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"82318735e638ef2cf41a3c2d","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-82318735e638ef2cf41a3c2d: done: 0.010657 sec","job_status":"done","queue_duration_s":56.424777,"scheduling_latency_s":56.424791,"redis_calls":10,"redis_duration_s":0.001433,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000728,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010657,"completed_at":"2026-09-03T10:16:59.445Z","load_balancing_strategy":"primary","db_duration_s":0.000913,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2035gitlab # [ 240.470637] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6c7591affd65425d49df9a48","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6c7591affd65425d49df9a48: start","job_status":"start","queue_duration_s":56.43632,"scheduling_latency_s":56.436333}2036gitlab # [ 240.471527] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6c7591affd65425d49df9a48","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6c7591affd65425d49df9a48: done: 0.010395 sec","job_status":"done","queue_duration_s":56.43632,"scheduling_latency_s":56.436333,"redis_calls":10,"redis_duration_s":0.001252,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010395,"completed_at":"2026-09-03T10:16:59.456Z","load_balancing_strategy":"primary","db_duration_s":0.000939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2037gitlab # [ 240.491069] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fe0b9703135b9288e264f199","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fe0b9703135b9288e264f199: start","job_status":"start","queue_duration_s":56.447611,"scheduling_latency_s":56.447625}2038gitlab # [ 240.502030] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fe0b9703135b9288e264f199","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fe0b9703135b9288e264f199: done: 0.010423 sec","job_status":"done","queue_duration_s":56.447611,"scheduling_latency_s":56.447625,"redis_calls":10,"redis_duration_s":0.001251,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000742,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010423,"completed_at":"2026-09-03T10:16:59.468Z","load_balancing_strategy":"primary","db_duration_s":0.000978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2039gitlab # [ 240.564694] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6522d994832dcfcd7584cf92","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6522d994832dcfcd7584cf92: start","job_status":"start","queue_duration_s":56.458819,"scheduling_latency_s":56.458832}2040gitlab # [ 240.565519] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6522d994832dcfcd7584cf92","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6522d994832dcfcd7584cf92: done: 0.010407 sec","job_status":"done","queue_duration_s":56.458819,"scheduling_latency_s":56.458832,"redis_calls":10,"redis_duration_s":0.001281,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000775,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010407,"completed_at":"2026-09-03T10:16:59.479Z","load_balancing_strategy":"primary","db_duration_s":0.000902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2041gitlab # [ 240.579687] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"76bcac6bebf648edb65807f1","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-76bcac6bebf648edb65807f1: start","job_status":"start","queue_duration_s":56.470089,"scheduling_latency_s":56.470102}2042gitlab # [ 240.590601] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"76bcac6bebf648edb65807f1","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-76bcac6bebf648edb65807f1: done: 0.010457 sec","job_status":"done","queue_duration_s":56.470089,"scheduling_latency_s":56.470102,"redis_calls":10,"redis_duration_s":0.0011949999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000703,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010457,"completed_at":"2026-09-03T10:16:59.490Z","load_balancing_strategy":"primary","db_duration_s":0.000918,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2043gitlab # [ 240.645626] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d5bbb31bb1f5734dae3f03e8","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d5bbb31bb1f5734dae3f03e8: start","job_status":"start","queue_duration_s":56.481313,"scheduling_latency_s":56.481326}2044gitlab # [ 240.646470] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d5bbb31bb1f5734dae3f03e8","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d5bbb31bb1f5734dae3f03e8: done: 0.010324 sec","job_status":"done","queue_duration_s":56.481313,"scheduling_latency_s":56.481326,"redis_calls":10,"redis_duration_s":0.001218,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00073,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010324,"completed_at":"2026-09-03T10:16:59.501Z","load_balancing_strategy":"primary","db_duration_s":0.000904,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2045gitlab # [ 240.700981] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c247bbc3944052c256cc6e6a","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c247bbc3944052c256cc6e6a: start","job_status":"start","queue_duration_s":56.492811,"scheduling_latency_s":56.492825}2046gitlab # [ 240.701768] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c247bbc3944052c256cc6e6a","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c247bbc3944052c256cc6e6a: done: 0.010619 sec","job_status":"done","queue_duration_s":56.492811,"scheduling_latency_s":56.492825,"redis_calls":10,"redis_duration_s":0.0012109999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000708,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010619,"completed_at":"2026-09-03T10:16:59.513Z","load_balancing_strategy":"primary","db_duration_s":0.001034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047gitlab # [ 240.713452] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2effa2dfe9b30a6c602da117","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2effa2dfe9b30a6c602da117: start","job_status":"start","queue_duration_s":56.504307,"scheduling_latency_s":56.504319}2048gitlab # [ 240.724105] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2effa2dfe9b30a6c602da117","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2effa2dfe9b30a6c602da117: done: 0.011749 sec","job_status":"done","queue_duration_s":56.504307,"scheduling_latency_s":56.504319,"redis_calls":10,"redis_duration_s":0.002356,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011749,"completed_at":"2026-09-03T10:16:59.526Z","load_balancing_strategy":"primary","db_duration_s":0.001012,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049gitlab # [ 240.758603] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"73a05e49355771a00852d6b9","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-73a05e49355771a00852d6b9: start","job_status":"start","queue_duration_s":56.516952,"scheduling_latency_s":56.516965}2050gitlab # [ 240.806834] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"73a05e49355771a00852d6b9","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-73a05e49355771a00852d6b9: done: 0.010535 sec","job_status":"done","queue_duration_s":56.516952,"scheduling_latency_s":56.516965,"redis_calls":10,"redis_duration_s":0.00121,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000717,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010535,"completed_at":"2026-09-03T10:16:59.537Z","load_balancing_strategy":"primary","db_duration_s":0.000957,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2051gitlab # [ 240.807916] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fcc618c4f69a29c0a1d85f46","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fcc618c4f69a29c0a1d85f46: start","job_status":"start","queue_duration_s":56.528755,"scheduling_latency_s":56.528769}2052gitlab # [ 240.815167] gitaly[920]: time="2026-09-03T10:17:00.746Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:10.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:00.740" grpc.time_ms=6.04 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2053gitlab # [ 240.827313] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fcc618c4f69a29c0a1d85f46","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fcc618c4f69a29c0a1d85f46: done: 0.012008 sec","job_status":"done","queue_duration_s":56.528755,"scheduling_latency_s":56.528769,"redis_calls":10,"redis_duration_s":0.0018050000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012008,"completed_at":"2026-09-03T10:16:59.550Z","load_balancing_strategy":"primary","db_duration_s":0.002053,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2054gitlab # [ 240.877053] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"00abdc93fb4cd7f6ed70cda2","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-00abdc93fb4cd7f6ed70cda2: start","job_status":"start","queue_duration_s":56.541731,"scheduling_latency_s":56.541745}2055gitlab # [ 240.877780] gitaly[920]: time="2026-09-03T10:17:00.755Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:17:10.019" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:00.749" grpc.time_ms=5.404 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2056gitlab # [ 240.886091] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"00abdc93fb4cd7f6ed70cda2","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-00abdc93fb4cd7f6ed70cda2: done: 0.011603 sec","job_status":"done","queue_duration_s":56.541731,"scheduling_latency_s":56.541745,"redis_calls":10,"redis_duration_s":0.001827,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011603,"completed_at":"2026-09-03T10:16:59.563Z","load_balancing_strategy":"primary","db_duration_s":0.001395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057gitlab # [ 240.933647] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f25053da2f2b236d7cb00d94","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f25053da2f2b236d7cb00d94: start","job_status":"start","queue_duration_s":56.554302,"scheduling_latency_s":56.554315}2058gitlab # [ 240.934271] gitaly[920]: time="2026-09-03T10:17:00.788Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-03T10:17:10.051" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:00.781" grpc.time_ms=6.282 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2059gitlab # [ 240.978273] gitaly[920]: time="2026-09-03T10:17:00.812Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:00.812" grpc.time_ms=0.284 pid=920 span.kind=server system=grpc2060gitlab # [ 240.980065] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f25053da2f2b236d7cb00d94","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f25053da2f2b236d7cb00d94: done: 0.017815 sec","job_status":"done","queue_duration_s":56.554302,"scheduling_latency_s":56.554315,"redis_calls":10,"redis_duration_s":0.008024,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.007258,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017815,"completed_at":"2026-09-03T10:16:59.582Z","load_balancing_strategy":"primary","db_duration_s":0.001208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2061gitlab # [ 240.988113] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5fd3ef373a6f6ce64cebaa44","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5fd3ef373a6f6ce64cebaa44: start","job_status":"start","queue_duration_s":56.573104,"scheduling_latency_s":56.573118}2062gitlab # [ 240.999301] gitaly[920]: time="2026-09-03T10:17:00.827Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:00.827" grpc.time_ms=0.249 pid=920 span.kind=server system=grpc2063gitlab # [ 241.007888] gitaly[920]: time="2026-09-03T10:17:00.836Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 diskcache=3be7c752-a1f0-4a14-8645-be3f70623d63 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:17:10.063" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:00.793" pid=920 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2064gitlab # [ 241.016937] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5fd3ef373a6f6ce64cebaa44","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5fd3ef373a6f6ce64cebaa44: done: 0.011015 sec","job_status":"done","queue_duration_s":56.573104,"scheduling_latency_s":56.573118,"redis_calls":10,"redis_duration_s":0.001748,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011015,"completed_at":"2026-09-03T10:16:59.594Z","load_balancing_strategy":"primary","db_duration_s":0.001004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab # [ 241.076289] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e95d019fe55929244db6900a","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e95d019fe55929244db6900a: start","job_status":"start","queue_duration_s":56.585149,"scheduling_latency_s":56.585162}2066gitlab # [ 241.076882] gitaly[920]: time="2026-09-03T10:17:00.836Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=41 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4901 command.oublock=8 command.real_time_ms=36 command.spawn_token_fork_ms=0 command.system_time_ms=24 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:17:10.063" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:00.793" grpc.time_ms=42.752 pid=920 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2067gitlab # [ 241.077634] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.606Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e95d019fe55929244db6900a","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e95d019fe55929244db6900a: done: 0.010951 sec","job_status":"done","queue_duration_s":56.585149,"scheduling_latency_s":56.585162,"redis_calls":10,"redis_duration_s":0.001769,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010951,"completed_at":"2026-09-03T10:16:59.606Z","load_balancing_strategy":"primary","db_duration_s":0.000962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2068gitlab # [ 241.137704] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"eebdc9e00d14733ffefd19eb","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-eebdc9e00d14733ffefd19eb: start","job_status":"start","queue_duration_s":56.597106,"scheduling_latency_s":56.59712}2069gitlab # [ 241.138626] gitaly[920]: time="2026-09-03T10:17:00.846Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:17:10.116" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:00.846" grpc.time_ms=0.085 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2070gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'2071gitlab # [ 241.139273] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"eebdc9e00d14733ffefd19eb","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-eebdc9e00d14733ffefd19eb: done: 0.011101 sec","job_status":"done","queue_duration_s":56.597106,"scheduling_latency_s":56.59712,"redis_calls":10,"redis_duration_s":0.001833,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011101,"completed_at":"2026-09-03T10:16:59.618Z","load_balancing_strategy":"primary","db_duration_s":0.000944,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)2073gitlab # [ 241.195643] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b0d91468fc7711755b2559ac","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b0d91468fc7711755b2559ac: start","job_status":"start","queue_duration_s":56.609199,"scheduling_latency_s":56.609212}2074gitlab # [ 241.196514] gitaly[920]: time="2026-09-03T10:17:00.856Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=934 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-03T10:17:10.119" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:00.850" grpc.time_ms=6.642 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2075gitlab # [ 241.197165] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b0d91468fc7711755b2559ac","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b0d91468fc7711755b2559ac: done: 0.010516 sec","job_status":"done","queue_duration_s":56.609199,"scheduling_latency_s":56.609212,"redis_calls":10,"redis_duration_s":0.0012950000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000796,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010516,"completed_at":"2026-09-03T10:16:59.629Z","load_balancing_strategy":"primary","db_duration_s":0.000949,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076gitlab # [ 241.248113] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b78ba80d8461768dde0925e0","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b78ba80d8461768dde0925e0: start","job_status":"start","queue_duration_s":56.620568,"scheduling_latency_s":56.620582}2077gitlab # [ 241.248911] gitaly[920]: time="2026-09-03T10:17:00.907Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-03T10:17:30.003" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:00.903" grpc.time_ms=3.854 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2078gitlab # [ 241.249647] nixbot[2597]: INFO: ::ffff:127.0.0.1:47386 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK2079gitlab # [ 241.250102] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b78ba80d8461768dde0925e0","created_at":"2026-09-03T10:16:02.539Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-64da471d2c8b444a","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"354d3ace18b1d6df7e3daa9f748fa853","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.010Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b78ba80d8461768dde0925e0: done: 0.010432 sec","job_status":"done","queue_duration_s":56.620568,"scheduling_latency_s":56.620582,"redis_calls":10,"redis_duration_s":0.001241,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010432,"completed_at":"2026-09-03T10:16:59.641Z","load_balancing_strategy":"primary","db_duration_s":0.000972,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2080gitlab # [ 241.297168] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a6122c95edb3b4af461843f7","created_at":"2026-09-03T10:16:03.032Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"338a56b164b34f4dc24b560ecae219a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:16:03.043Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a6122c95edb3b4af461843f7: start","job_status":"start","queue_duration_s":56.598905,"scheduling_latency_s":56.598944}2081gitlab # [ 241.297931] gitaly[920]: time="2026-09-03T10:17:00.926Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-03T10:17:10.195" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:00.926" grpc.time_ms=0.176 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2082gitlab # [ 241.300864] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.663Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a6122c95edb3b4af461843f7","created_at":"2026-09-03T10:16:03.032Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"338a56b164b34f4dc24b560ecae219a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:16:03.043Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a6122c95edb3b4af461843f7: done: 0.021833 sec","job_status":"done","queue_duration_s":56.598905,"scheduling_latency_s":56.598944,"redis_calls":11,"redis_duration_s":0.001604,"redis_read_bytes":9,"redis_write_bytes":1449,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000686,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":708,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021833,"completed_at":"2026-09-03T10:16:59.663Z","load_balancing_strategy":"primary","db_duration_s":0.004156,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2083gitlab # [ 241.341716] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b808f8950dbf1014ba5bd5ab","created_at":"2026-09-03T10:16:03.117Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c7a827a3a2450e4cb50e5c6250fe69e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.126Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b808f8950dbf1014ba5bd5ab: start","job_status":"start","queue_duration_s":56.538603,"scheduling_latency_s":56.538618}2084gitlab # [ 241.382241] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.673Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2085gitlab # [ 241.383374] gitaly[920]: time="2026-09-03T10:17:00.931Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-03T10:17:10.198" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-03T10:17:00.928" grpc.time_ms=3.563 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2086gitlab # [ 241.383981] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.677Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b808f8950dbf1014ba5bd5ab","created_at":"2026-09-03T10:16:03.117Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-bff0f31b93004ad1","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c7a827a3a2450e4cb50e5c6250fe69e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.126Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b808f8950dbf1014ba5bd5ab: done: 0.012752 sec","job_status":"done","queue_duration_s":56.538603,"scheduling_latency_s":56.538618,"redis_calls":11,"redis_duration_s":0.001389,"redis_read_bytes":9,"redis_write_bytes":1589,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":768,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012752,"completed_at":"2026-09-03T10:16:59.677Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087gitlab # [ 241.406600] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fe16abbcf804cc8d9ee44c2","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fe16abbcf804cc8d9ee44c2: start","job_status":"start","queue_duration_s":56.425764,"scheduling_latency_s":56.425777}2088gitlab # [ 241.417966] gitaly[920]: time="2026-09-03T10:17:00.945Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:10.211" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:00.941" grpc.time_ms=4.502 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2089gitlab # [ 241.463868] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fe16abbcf804cc8d9ee44c2","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fe16abbcf804cc8d9ee44c2: done: 0.017587 sec","job_status":"done","queue_duration_s":56.425764,"scheduling_latency_s":56.425777,"redis_calls":23,"redis_duration_s":0.002783,"redis_read_bytes":20,"redis_write_bytes":2604,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001212,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1447,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017587,"completed_at":"2026-09-03T10:16:59.696Z","load_balancing_strategy":"primary","db_duration_s":0.000984,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090gitlab # [ 241.466668] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee634eff68572704bd043d57","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee634eff68572704bd043d57: start","job_status":"start","queue_duration_s":56.444183,"scheduling_latency_s":56.444198}2091gitlab # [ 241.510089] gitaly[920]: time="2026-09-03T10:17:00.954Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:10.218" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:00.948" grpc.time_ms=5.046 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2092gitlab # [ 241.510765] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee634eff68572704bd043d57","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee634eff68572704bd043d57: done: 0.009075 sec","job_status":"done","queue_duration_s":56.444183,"scheduling_latency_s":56.444198,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009075,"completed_at":"2026-09-03T10:16:59.705Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2093gitlab # [ 241.524854] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1cea36a61ab26f0fd0190ec","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1cea36a61ab26f0fd0190ec: start","job_status":"start","queue_duration_s":56.454064,"scheduling_latency_s":56.454076}2094gitlab # [ 241.536494] gitaly[920]: time="2026-09-03T10:17:00.967Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=956 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-03T10:17:30.060" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:00.960" grpc.time_ms=7.497 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2095gitlab # [ 241.550258] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1cea36a61ab26f0fd0190ec","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1cea36a61ab26f0fd0190ec: done: 0.009817 sec","job_status":"done","queue_duration_s":56.454064,"scheduling_latency_s":56.454076,"redis_calls":13,"redis_duration_s":0.002139,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001423,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009817,"completed_at":"2026-09-03T10:16:59.716Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2096gitlab # [ 241.608687] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"602f1dbb5b3f97668370ec25","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-602f1dbb5b3f97668370ec25: start","job_status":"start","queue_duration_s":56.464677,"scheduling_latency_s":56.46469}2097gitlab # [ 241.609463] gitaly[920]: time="2026-09-03T10:17:01.257Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-03T10:17:11.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.254" grpc.time_ms=2.727 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098gitlab # [ 241.610144] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"602f1dbb5b3f97668370ec25","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-602f1dbb5b3f97668370ec25: done: 0.008913 sec","job_status":"done","queue_duration_s":56.464677,"scheduling_latency_s":56.46469,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008913,"completed_at":"2026-09-03T10:16:59.726Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2099gitlab # [ 241.644951] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"747c46899d1cac48aeb5b221","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-747c46899d1cac48aeb5b221: start","job_status":"start","queue_duration_s":56.47446,"scheduling_latency_s":56.474472}2100gitlab # [ 241.656668] gitaly[920]: time="2026-09-03T10:17:01.303Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:11.048" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:01.298" grpc.time_ms=4.915 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2101gitlab # [ 241.669680] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"747c46899d1cac48aeb5b221","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-747c46899d1cac48aeb5b221: done: 0.00891 sec","job_status":"done","queue_duration_s":56.47446,"scheduling_latency_s":56.474472,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00891,"completed_at":"2026-09-03T10:16:59.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102gitlab # [ 241.711232] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e4c683028170191231a579","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e4c683028170191231a579: start","job_status":"start","queue_duration_s":56.484237,"scheduling_latency_s":56.484249}2103gitlab # [ 241.716128] gitaly[920]: time="2026-09-03T10:17:01.312Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:17:11.056" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:01.307" grpc.time_ms=4.745 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2104gitlab # [ 241.728797] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e4c683028170191231a579","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e4c683028170191231a579: done: 0.008973 sec","job_status":"done","queue_duration_s":56.484237,"scheduling_latency_s":56.484249,"redis_calls":13,"redis_duration_s":0.0014529999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008973,"completed_at":"2026-09-03T10:16:59.745Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2105gitlab # [ 241.791943] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"021866474ea3978ecb499d33","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-021866474ea3978ecb499d33: start","job_status":"start","queue_duration_s":56.49407,"scheduling_latency_s":56.494084}2106gitlab # [ 241.792767] gitaly[920]: time="2026-09-03T10:17:01.326Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-03T10:17:11.071" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:01.321" grpc.time_ms=5.222 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2107gitlab # [ 241.793501] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"021866474ea3978ecb499d33","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-021866474ea3978ecb499d33: done: 0.008931 sec","job_status":"done","queue_duration_s":56.49407,"scheduling_latency_s":56.494084,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008931,"completed_at":"2026-09-03T10:16:59.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2108gitlab # [ 241.838363] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bd89b14f78b9dc0579f8b13","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bd89b14f78b9dc0579f8b13: start","job_status":"start","queue_duration_s":56.503857,"scheduling_latency_s":56.503869}2109gitlab # [ 241.839107] gitaly[920]: time="2026-09-03T10:17:01.334Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-03T10:17:31.028" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.328" grpc.time_ms=5.754 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2110gitlab # [ 241.847339] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bd89b14f78b9dc0579f8b13","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bd89b14f78b9dc0579f8b13: done: 0.008982 sec","job_status":"done","queue_duration_s":56.503857,"scheduling_latency_s":56.503869,"redis_calls":13,"redis_duration_s":0.001458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000469,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008982,"completed_at":"2026-09-03T10:16:59.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2111gitlab # [ 241.900149] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d9b10d12facdf7a73152c38","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d9b10d12facdf7a73152c38: start","job_status":"start","queue_duration_s":56.513788,"scheduling_latency_s":56.513801}2112gitlab # [ 241.900882] gitaly[920]: time="2026-09-03T10:17:01.342Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-03T10:17:11.089" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-03T10:17:01.339" grpc.time_ms=2.912 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2113gitlab # [ 241.906256] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d9b10d12facdf7a73152c38","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d9b10d12facdf7a73152c38: done: 0.009028 sec","job_status":"done","queue_duration_s":56.513788,"scheduling_latency_s":56.513801,"redis_calls":13,"redis_duration_s":0.001451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009028,"completed_at":"2026-09-03T10:16:59.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2114gitlab # [ 241.949684] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"791644375c1bd054ffd6ecc7","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-791644375c1bd054ffd6ecc7: start","job_status":"start","queue_duration_s":56.523617,"scheduling_latency_s":56.52363}2115gitlab # [ 241.952619] gitaly[920]: time="2026-09-03T10:17:01.352Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:11.097" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:01.347" grpc.time_ms=4.727 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2116gitlab # [ 241.964507] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"791644375c1bd054ffd6ecc7","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-791644375c1bd054ffd6ecc7: done: 0.008949 sec","job_status":"done","queue_duration_s":56.523617,"scheduling_latency_s":56.52363,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008949,"completed_at":"2026-09-03T10:16:59.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2117gitlab # [ 242.031092] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91453552990fafee800bca43","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91453552990fafee800bca43: start","job_status":"start","queue_duration_s":56.533342,"scheduling_latency_s":56.533355}2118gitlab # [ 242.031834] gitaly[920]: time="2026-09-03T10:17:01.405Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-03T10:17:11.154" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.405" grpc.time_ms=0.583 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2119gitlab # [ 242.032453] gitaly[920]: time="2026-09-03T10:17:01.617Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:01.617" grpc.time_ms=0.234 pid=920 span.kind=server system=grpc2120gitlab # [ 242.033079] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91453552990fafee800bca43","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91453552990fafee800bca43: done: 0.008977 sec","job_status":"done","queue_duration_s":56.533342,"scheduling_latency_s":56.533355,"redis_calls":13,"redis_duration_s":0.001443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008977,"completed_at":"2026-09-03T10:16:59.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121gitlab # [ 242.065941] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00599111b7594bdfa6119abf","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00599111b7594bdfa6119abf: start","job_status":"start","queue_duration_s":56.543134,"scheduling_latency_s":56.543149}2122gitlab # [ 242.077557] gitaly[920]: time="2026-09-03T10:17:01.639Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:01.638" grpc.time_ms=0.86 pid=920 span.kind=server system=grpc2123gitlab # [ 242.123239] gitaly[920]: time="2026-09-03T10:17:01.656Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:01.656" grpc.time_ms=0.222 pid=920 span.kind=server system=grpc2124gitlab # [ 242.123880] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00599111b7594bdfa6119abf","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00599111b7594bdfa6119abf: done: 0.008937 sec","job_status":"done","queue_duration_s":56.543134,"scheduling_latency_s":56.543149,"redis_calls":13,"redis_duration_s":0.0015830000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008937,"completed_at":"2026-09-03T10:16:59.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125gitlab # [ 242.129669] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86a5fd5c6426da9a4fa00661","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86a5fd5c6426da9a4fa00661: start","job_status":"start","queue_duration_s":56.552931,"scheduling_latency_s":56.552943}2126gitlab # [ 242.170180] gitaly[920]: time="2026-09-03T10:17:01.665Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X diskcache=d3ed9c50-02e7-4f68-9198-130c4fb70947 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:31.298" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:01.598" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2127gitlab # [ 242.170841] gitaly[920]: time="2026-09-03T10:17:01.665Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7216 command.oublock=0 command.real_time_ms=60 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=22 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:31.298" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:01.598" grpc.time_ms=66.554 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2128gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'2129gitlab # [ 242.171528] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86a5fd5c6426da9a4fa00661","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86a5fd5c6426da9a4fa00661: done: 0.008949 sec","job_status":"done","queue_duration_s":56.552931,"scheduling_latency_s":56.552943,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008949,"completed_at":"2026-09-03T10:16:59.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2130gitlab # [ 242.195785] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2103809c1c45b2d2cd38b0ff","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2103809c1c45b2d2cd38b0ff: start","job_status":"start","queue_duration_s":56.562693,"scheduling_latency_s":56.562706}2131gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.04 seconds)2132gitlab # [ 242.206786] gitaly[920]: time="2026-09-03T10:17:01.733Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:17:31.431" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.732" grpc.time_ms=0.571 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2133gitlab # [ 242.217936] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2103809c1c45b2d2cd38b0ff","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2103809c1c45b2d2cd38b0ff: done: 0.00886 sec","job_status":"done","queue_duration_s":56.562693,"scheduling_latency_s":56.562706,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00886,"completed_at":"2026-09-03T10:16:59.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2134gitlab # [ 242.283892] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89e8b7b5041f39ceee13d021","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89e8b7b5041f39ceee13d021: start","job_status":"start","queue_duration_s":56.572407,"scheduling_latency_s":56.572419}2135gitlab # [ 242.284651] gitaly[920]: time="2026-09-03T10:17:01.755Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:31.454" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.755" grpc.time_ms=0.434 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2136gitlab # [ 242.285215] gitaly[920]: time="2026-09-03T10:17:01.795Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-03T16:17:01.793" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:01.793" grpc.time_ms=2.103 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2137gitlab # [ 242.285995] nixbot[2597]: INFO: ::ffff:127.0.0.1:47398 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK2138gitlab # [ 242.286551] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89e8b7b5041f39ceee13d021","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89e8b7b5041f39ceee13d021: done: 0.008916 sec","job_status":"done","queue_duration_s":56.572407,"scheduling_latency_s":56.572419,"redis_calls":13,"redis_duration_s":0.001449,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008916,"completed_at":"2026-09-03T10:16:59.833Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139gitlab # [ 242.354256] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"737389e54f5c19da4960bdc2","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-737389e54f5c19da4960bdc2: start","job_status":"start","queue_duration_s":56.582164,"scheduling_latency_s":56.582177}2140gitlab # [ 242.355063] gitaly[920]: time="2026-09-03T10:17:01.805Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=656 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-03T10:17:31.499" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.800" grpc.time_ms=5.142 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2141gitlab # [ 242.355767] gitaly[920]: time="2026-09-03T10:17:01.857Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 diskcache=2b87666f-2ec7-4265-9eb5-24356c69cfb3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-03T16:17:01.838" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.838" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2142gitlab # [ 242.356554] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"737389e54f5c19da4960bdc2","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-737389e54f5c19da4960bdc2: done: 0.009021 sec","job_status":"done","queue_duration_s":56.582164,"scheduling_latency_s":56.582177,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009021,"completed_at":"2026-09-03T10:16:59.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143gitlab # [ 242.398073] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba5469b845aba9170275cf7f","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba5469b845aba9170275cf7f: start","job_status":"start","queue_duration_s":56.592136,"scheduling_latency_s":56.592149}2144gitlab # [ 242.398804] gitaly[920]: time="2026-09-03T10:17:01.857Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1279 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-03T16:17:01.838" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.838" grpc.time_ms=18.92 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2145gitlab # [ 242.443901] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba5469b845aba9170275cf7f","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba5469b845aba9170275cf7f: done: 0.008918 sec","job_status":"done","queue_duration_s":56.592136,"scheduling_latency_s":56.592149,"redis_calls":13,"redis_duration_s":0.001475,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008918,"completed_at":"2026-09-03T10:16:59.853Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2146gitlab # [ 242.444987] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37d92c399433f70848e1308e","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37d92c399433f70848e1308e: start","job_status":"start","queue_duration_s":56.601915,"scheduling_latency_s":56.601928}2147gitlab # [ 242.483745] gitaly[920]: time="2026-09-03T10:17:01.909Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-03T10:17:31.004" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.904" grpc.time_ms=5.245 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2148gitlab # [ 242.484597] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37d92c399433f70848e1308e","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37d92c399433f70848e1308e: done: 0.008941 sec","job_status":"done","queue_duration_s":56.601915,"scheduling_latency_s":56.601928,"redis_calls":13,"redis_duration_s":0.0014529999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000463,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008941,"completed_at":"2026-09-03T10:16:59.863Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2149gitlab # [ 242.523821] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1716c4b20bf5e76100a2e02","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1716c4b20bf5e76100a2e02: start","job_status":"start","queue_duration_s":56.611715,"scheduling_latency_s":56.611735}2150gitlab # [ 242.524708] gitaly[920]: time="2026-09-03T10:17:01.932Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:31.032" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:01.932" grpc.time_ms=0.413 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2151gitlab # [ 242.525467] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1716c4b20bf5e76100a2e02","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1716c4b20bf5e76100a2e02: done: 0.00903 sec","job_status":"done","queue_duration_s":56.611715,"scheduling_latency_s":56.611735,"redis_calls":13,"redis_duration_s":0.0014579999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00903,"completed_at":"2026-09-03T10:16:59.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2152gitlab # [ 242.557788] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b57b0b6f02cec62d5b1905bf","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b57b0b6f02cec62d5b1905bf: start","job_status":"start","queue_duration_s":56.621544,"scheduling_latency_s":56.621557}2153gitlab # [ 242.605670] gitaly[920]: time="2026-09-03T10:17:01.939Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-03T10:17:11.204" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:01.934" grpc.time_ms=5.059 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2154gitlab # [ 242.606497] nixbot[2597]: INFO: ::ffff:127.0.0.1:47414 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2155gitlab # [ 242.606920] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b57b0b6f02cec62d5b1905bf","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b57b0b6f02cec62d5b1905bf: done: 0.008986 sec","job_status":"done","queue_duration_s":56.621544,"scheduling_latency_s":56.621557,"redis_calls":13,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008986,"completed_at":"2026-09-03T10:16:59.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156gitlab # [ 242.616599] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5400bae02756ef2bb157eea8","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5400bae02756ef2bb157eea8: start","job_status":"start","queue_duration_s":56.631319,"scheduling_latency_s":56.631332}2157gitlab # [ 242.628357] gitaly[920]: time="2026-09-03T10:17:02.008Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:17:32.007" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.007" grpc.time_ms=0.672 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2158gitlab # [ 242.639983] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5400bae02756ef2bb157eea8","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5400bae02756ef2bb157eea8: done: 0.008949 sec","job_status":"done","queue_duration_s":56.631319,"scheduling_latency_s":56.631332,"redis_calls":13,"redis_duration_s":0.001422,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008949,"completed_at":"2026-09-03T10:16:59.892Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2159gitlab # [ 242.688627] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d45375dc493ece4bb3ae1ac","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d45375dc493ece4bb3ae1ac: start","job_status":"start","queue_duration_s":56.641069,"scheduling_latency_s":56.641082}2160gitlab # [ 242.689513] gitaly[920]: time="2026-09-03T10:17:02.012Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=37 command.oublock=0 command.real_time_ms=0 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-03T10:17:32.010" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.010" grpc.time_ms=1.626 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2161gitlab # [ 242.701480] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 582162gitlab # [ 242.703065] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d45375dc493ece4bb3ae1ac","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d45375dc493ece4bb3ae1ac: done: 0.00899 sec","job_status":"done","queue_duration_s":56.641069,"scheduling_latency_s":56.641082,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000479,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00899,"completed_at":"2026-09-03T10:16:59.902Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2163gitlab # [ 242.753080] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b0aa0818072af60616181b4","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b0aa0818072af60616181b4: start","job_status":"start","queue_duration_s":56.65095,"scheduling_latency_s":56.650963}2164gitlab # [ 242.753800] gitaly[920]: time="2026-09-03T10:17:02.065Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:12.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:02.058" grpc.time_ms=7.415 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2165gitlab # [ 242.798801] nixbot[2597]: {"ts": "2026-09-03T10:17:02.729Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2166gitlab # [ 242.799691] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b0aa0818072af60616181b4","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b0aa0818072af60616181b4: done: 0.009084 sec","job_status":"done","queue_duration_s":56.65095,"scheduling_latency_s":56.650963,"redis_calls":13,"redis_duration_s":0.001631,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009084,"completed_at":"2026-09-03T10:16:59.912Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2167gitlab # [ 242.800656] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2eca1137cee5c2a461bd75d","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2eca1137cee5c2a461bd75d: start","job_status":"start","queue_duration_s":56.660834,"scheduling_latency_s":56.660847}2168gitlab # [ 242.809642] gitlab-workhorse[826]: time="2026-09-03T10:17:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672169gitlab # [ 242.811197] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 572170gitlab # [ 242.813234] gitaly[920]: time="2026-09-03T10:17:02.464Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-03T10:17:32.455" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.455" grpc.time_ms=8.462 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2171gitlab # [ 242.826655] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2eca1137cee5c2a461bd75d","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2eca1137cee5c2a461bd75d: done: 0.009108 sec","job_status":"done","queue_duration_s":56.660834,"scheduling_latency_s":56.660847,"redis_calls":13,"redis_duration_s":0.0016769999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009108,"completed_at":"2026-09-03T10:16:59.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172gitlab # [ 242.873745] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5169e20fa76270c2938343ac","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5169e20fa76270c2938343ac: start","job_status":"start","queue_duration_s":56.67082,"scheduling_latency_s":56.670833}2173gitlab # [ 242.874992] gitaly[920]: time="2026-09-03T10:17:02.473Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9M8RN7T7TW2X3MGAKXQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-03T10:17:32.465" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.465" grpc.time_ms=7.481 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2174gitlab # [ 242.917511] gitaly[920]: time="2026-09-03T10:17:02.616Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W diskcache=801e4662-2e85-48de-ac56-1bea791e60a0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-03T16:17:02.604" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.604" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2175gitlab # [ 242.918416] nixbot[2597]: {"ts": "2026-09-03T10:17:02.839Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 \"HTTP/1.1 201 Created\""}2176gitlab # [ 242.918865] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5169e20fa76270c2938343ac","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5169e20fa76270c2938343ac: done: 0.008852 sec","job_status":"done","queue_duration_s":56.67082,"scheduling_latency_s":56.670833,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008852,"completed_at":"2026-09-03T10:16:59.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2177gitlab # [ 242.963590] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e03480a2a49bcddc2acea53c","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e03480a2a49bcddc2acea53c: start","job_status":"start","queue_duration_s":56.680573,"scheduling_latency_s":56.680586}2178gitlab # [ 242.964480] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 1012179gitlab # [ 242.964890] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1042180gitlab # [ 242.965847] gitaly[920]: time="2026-09-03T10:17:02.616Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-03T16:17:02.604" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.604" grpc.time_ms=12.281 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2181gitlab # [ 242.966585] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e03480a2a49bcddc2acea53c","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e03480a2a49bcddc2acea53c: done: 0.008886 sec","job_status":"done","queue_duration_s":56.680573,"scheduling_latency_s":56.680586,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008886,"completed_at":"2026-09-03T10:16:59.942Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2182gitlab # [ 243.013182] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"764fb54593987b449b179d9c","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-764fb54593987b449b179d9c: start","job_status":"start","queue_duration_s":56.69033,"scheduling_latency_s":56.690343}2183gitlab # [ 243.013882] nixbot[2597]: {"ts": "2026-09-03T10:17:02.947Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 \"HTTP/1.1 201 Created\""}2184gitlab # [ 243.014693] gitaly[920]: time="2026-09-03T10:17:02.648Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:32.646" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.647" grpc.time_ms=0.563 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2185gitlab # [ 243.016523] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"764fb54593987b449b179d9c","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-764fb54593987b449b179d9c: done: 0.009101 sec","job_status":"done","queue_duration_s":56.69033,"scheduling_latency_s":56.690343,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009101,"completed_at":"2026-09-03T10:16:59.952Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2186gitlab # [ 243.059085] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67dd208556697e060c767052","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67dd208556697e060c767052: start","job_status":"start","queue_duration_s":56.700268,"scheduling_latency_s":56.70028}2187gitlab # [ 243.062211] gitaly[920]: time="2026-09-03T10:17:02.651Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB7W1HNXS7209S9A4HS4C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:02.644" grpc.time_ms=7.141 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2188gitlab # [ 243.074648] nixbot[2597]: {"ts": "2026-09-03T10:17:03.043Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 \"HTTP/1.1 201 Created\""}2189gitlab # [ 243.077066] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67dd208556697e060c767052","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67dd208556697e060c767052: done: 0.008948 sec","job_status":"done","queue_duration_s":56.700268,"scheduling_latency_s":56.70028,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008948,"completed_at":"2026-09-03T10:16:59.961Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2190gitlab # [ 243.148825] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f290cf498cdb3e49d7a3069","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f290cf498cdb3e49d7a3069: start","job_status":"start","queue_duration_s":56.710101,"scheduling_latency_s":56.710115}2191gitlab # [ 243.149718] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eedce16a998e57e3e500c437ccc385d7d5ac4a49 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 902192gitlab # [ 243.150129] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 372193gitlab # [ 243.150619] gitaly[920]: time="2026-09-03T10:17:02.655Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-03T10:17:12.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:02.650" grpc.time_ms=4.963 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2194gitlab # [ 243.151290] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f290cf498cdb3e49d7a3069","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f290cf498cdb3e49d7a3069: done: 0.008935 sec","job_status":"done","queue_duration_s":56.710101,"scheduling_latency_s":56.710115,"redis_calls":13,"redis_duration_s":0.0014589999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008935,"completed_at":"2026-09-03T10:16:59.971Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195gitlab # [ 243.204109] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"173bce802a16c593b774fb0d","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-173bce802a16c593b774fb0d: start","job_status":"start","queue_duration_s":56.719835,"scheduling_latency_s":56.719848}2196gitlab # [ 243.204862] nixbot[2597]: INFO: ::ffff:127.0.0.1:59962 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2197gitlab # [ 243.205575] gitaly[920]: time="2026-09-03T10:17:02.676Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:17:32.675" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.676" grpc.time_ms=0.707 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2198gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'2199gitlab # [ 243.206239] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"173bce802a16c593b774fb0d","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-173bce802a16c593b774fb0d: done: 0.009017 sec","job_status":"done","queue_duration_s":56.719835,"scheduling_latency_s":56.719848,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009017,"completed_at":"2026-09-03T10:16:59.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2200gitlab # [ 243.257214] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21b644cd4a5696a21bcfc043","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21b644cd4a5696a21bcfc043: start","job_status":"start","queue_duration_s":56.729644,"scheduling_latency_s":56.729657}2201gitlab # [ 243.257789] gitlab-workhorse[826]: time="2026-09-03T10:17:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672202gitlab # [ 243.260266] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 452203gitlab # [ 243.262184] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 442204gitlab # [ 243.262780] gitaly[920]: time="2026-09-03T10:17:02.685Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=63 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-03T10:17:32.683" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.683" grpc.time_ms=2.029 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2205gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 0.06 seconds)2206gitlab # [ 243.264550] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21b644cd4a5696a21bcfc043","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21b644cd4a5696a21bcfc043: done: 0.008925 sec","job_status":"done","queue_duration_s":56.729644,"scheduling_latency_s":56.729657,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008925,"completed_at":"2026-09-03T10:16:59.991Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2207gitlab # [ 243.299498] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:16:59.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c81015aeb94face41089c1e5","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c81015aeb94face41089c1e5: start","job_status":"start","queue_duration_s":56.739351,"scheduling_latency_s":56.739362}2208gitlab # [ 243.338816] gitaly[920]: time="2026-09-03T10:17:02.708Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:12.062" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:02.702" grpc.time_ms=5.909 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2209gitlab # [ 243.339646] gitaly[920]: time="2026-09-03T10:17:02.710Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB7XXBQ1M8MPDKNWWNM6G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:02.703" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2210gitlab # [ 243.340743] nixbot[2597]: INFO: ::ffff:127.0.0.1:59972 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK2211gitlab # [ 243.341257] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c81015aeb94face41089c1e5","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c81015aeb94face41089c1e5: done: 0.00898 sec","job_status":"done","queue_duration_s":56.739351,"scheduling_latency_s":56.739362,"redis_calls":13,"redis_duration_s":0.0014720000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00898,"completed_at":"2026-09-03T10:17:00.001Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212gitlab # [ 243.391873] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"199281efc9113072cc8c91a4","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-199281efc9113072cc8c91a4: start","job_status":"start","queue_duration_s":56.749276,"scheduling_latency_s":56.74929}2213gitlab # [ 243.392788] gitlab-workhorse[826]: time="2026-09-03T10:17:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672214gitlab # [ 243.393245] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 552215gitlab # [ 243.393726] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 352216gitlab # [ 243.395976] gitaly[920]: time="2026-09-03T10:17:02.710Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB7XXBQ1M8MPDKNWWNM6G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:02.703" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2217gitlab # [ 243.397133] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"199281efc9113072cc8c91a4","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-199281efc9113072cc8c91a4: done: 0.008972 sec","job_status":"done","queue_duration_s":56.749276,"scheduling_latency_s":56.74929,"redis_calls":13,"redis_duration_s":0.001522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008972,"completed_at":"2026-09-03T10:17:00.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2218gitlab # [ 243.459714] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"329d141cf64837ad4a09f073","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-329d141cf64837ad4a09f073: start","job_status":"start","queue_duration_s":56.759516,"scheduling_latency_s":56.759548}2219gitlab # [ 243.460959] gitaly[920]: time="2026-09-03T10:17:02.711Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB7XXBQ1M8MPDKNWWNM6G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:02.703" grpc.time_ms=7.866 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2220gitlab # [ 243.461705] gitlab-workhorse[826]: time="2026-09-03T10:17:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162221gitlab # [ 243.462133] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 442222gitlab # [ 243.462642] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"329d141cf64837ad4a09f073","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-329d141cf64837ad4a09f073: done: 0.01186 sec","job_status":"done","queue_duration_s":56.759516,"scheduling_latency_s":56.759548,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01186,"completed_at":"2026-09-03T10:17:00.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2223gitlab # [ 243.504780] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3644567f994c6486d9c689b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3644567f994c6486d9c689b: start","job_status":"start","queue_duration_s":56.772192,"scheduling_latency_s":56.772206}2224gitlab # [ 243.507924] gitaly[920]: time="2026-09-03T10:17:02.771Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB80HWDCMZMS1NZNM7TJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:32.770" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.770" grpc.time_ms=0.503 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2225gitlab # [ 243.550188] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3644567f994c6486d9c689b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3644567f994c6486d9c689b: done: 0.008962 sec","job_status":"done","queue_duration_s":56.772192,"scheduling_latency_s":56.772206,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008962,"completed_at":"2026-09-03T10:17:00.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2226gitlab # [ 243.594179] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cda6316f1806a2b4a3d5557b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cda6316f1806a2b4a3d5557b: start","job_status":"start","queue_duration_s":56.782085,"scheduling_latency_s":56.782098}2227gitlab # [ 243.594938] gitaly[920]: time="2026-09-03T10:17:02.808Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB80HWDCMZMS1NZNM7TJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:12.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:02.801" grpc.time_ms=7.066 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2228gitlab # [ 243.595678] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cda6316f1806a2b4a3d5557b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cda6316f1806a2b4a3d5557b: done: 0.009748 sec","job_status":"done","queue_duration_s":56.782085,"scheduling_latency_s":56.782098,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009748,"completed_at":"2026-09-03T10:17:00.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2229gitlab # [ 243.655312] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1107c29677f777bc3dcdcefb","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1107c29677f777bc3dcdcefb: start","job_status":"start","queue_duration_s":56.792752,"scheduling_latency_s":56.792766}2230gitlab # [ 243.655975] gitaly[920]: time="2026-09-03T10:17:02.879Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB83TBFHDZDMEG6AGQ3HX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:32.778" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.878" grpc.time_ms=0.499 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2231gitlab # [ 243.656616] nixbot[2597]: {"ts": "2026-09-03T10:17:03.619Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eba459b7c8f87c78f7a7240ca13a81922b9f7662 \"HTTP/1.1 201 Created\""}2232gitlab # [ 243.657100] nixbot[2597]: {"ts": "2026-09-03T10:17:03.626Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-eba459b7c8f8-4"}2233gitlab # [ 243.657605] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1107c29677f777bc3dcdcefb","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1107c29677f777bc3dcdcefb: done: 0.009187 sec","job_status":"done","queue_duration_s":56.792752,"scheduling_latency_s":56.792766,"redis_calls":13,"redis_duration_s":0.0017530000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009187,"completed_at":"2026-09-03T10:17:00.054Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2234gitlab # [ 243.718585] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"988364cb14f7f8730f1c08e9","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-988364cb14f7f8730f1c08e9: start","job_status":"start","queue_duration_s":56.802828,"scheduling_latency_s":56.802841}2235gitlab # [ 243.719602] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 1932236gitlab # [ 243.720060] gitaly[920]: time="2026-09-03T10:17:02.918Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB83TBFHDZDMEG6AGQ3HX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:12.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:02.913" grpc.time_ms=5.305 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2237gitlab # [ 243.720726] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"988364cb14f7f8730f1c08e9","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-988364cb14f7f8730f1c08e9: done: 0.00889 sec","job_status":"done","queue_duration_s":56.802828,"scheduling_latency_s":56.802841,"redis_calls":13,"redis_duration_s":0.001455,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00889,"completed_at":"2026-09-03T10:17:00.064Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2238gitlab # [ 243.771922] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0683946b01fc9b1f6c8dc27c","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0683946b01fc9b1f6c8dc27c: start","job_status":"start","queue_duration_s":56.812585,"scheduling_latency_s":56.812598}2239gitlab # [ 243.772626] gitaly[920]: time="2026-09-03T10:17:02.985Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB877QNJBD32E91QCZ2S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:32.184" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:02.985" grpc.time_ms=0.644 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2240gitlab # [ 243.773340] nixbot[2597]: {"ts": "2026-09-03T10:17:03.715Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-eba459b7c8f8-4", "argv0": "bwrap"}2241gitlab # [ 243.773745] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0683946b01fc9b1f6c8dc27c","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0683946b01fc9b1f6c8dc27c: done: 0.00901 sec","job_status":"done","queue_duration_s":56.812585,"scheduling_latency_s":56.812598,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00901,"completed_at":"2026-09-03T10:17:00.074Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2242gitlab # [ 243.836066] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f8c06bf982132e5f34f0f1b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f8c06bf982132e5f34f0f1b: start","job_status":"start","queue_duration_s":56.822458,"scheduling_latency_s":56.82247}2243gitlab # [ 243.836820] gitaly[920]: time="2026-09-03T10:17:03.010Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB877QNJBD32E91QCZ2S4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:13.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:03.005" grpc.time_ms=4.943 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2244gitlab # [ 243.837634] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f8c06bf982132e5f34f0f1b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f8c06bf982132e5f34f0f1b: done: 0.008831 sec","job_status":"done","queue_duration_s":56.822458,"scheduling_latency_s":56.82247,"redis_calls":13,"redis_duration_s":0.0014559999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000463,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008831,"completed_at":"2026-09-03T10:17:00.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2245gitlab # [ 243.879047] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29c8cc9415c620eaec8f68df","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29c8cc9415c620eaec8f68df: start","job_status":"start","queue_duration_s":56.832092,"scheduling_latency_s":56.832105}2246gitlab # [ 243.880127] gitaly[920]: time="2026-09-03T10:17:03.053Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=965 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-03T10:17:33.045" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:03.045" grpc.time_ms=8.148 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2247gitlab # [ 243.919697] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29c8cc9415c620eaec8f68df","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29c8cc9415c620eaec8f68df: done: 0.008933 sec","job_status":"done","queue_duration_s":56.832092,"scheduling_latency_s":56.832105,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008933,"completed_at":"2026-09-03T10:17:00.093Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248gitlab # [ 243.920303] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbb3f336df8859a017b0c7aa","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbb3f336df8859a017b0c7aa: start","job_status":"start","queue_duration_s":56.841824,"scheduling_latency_s":56.841837}2249gitlab # [ 243.930753] gitaly[920]: time="2026-09-03T10:17:03.064Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9QD2972HN4KTYCB33A5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-03T10:17:33.055" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:03.055" grpc.time_ms=8.467 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2250gitlab # [ 243.943100] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbb3f336df8859a017b0c7aa","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbb3f336df8859a017b0c7aa: done: 0.009 sec","job_status":"done","queue_duration_s":56.841824,"scheduling_latency_s":56.841837,"redis_calls":13,"redis_duration_s":0.0015240000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009,"completed_at":"2026-09-03T10:17:00.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2251gitlab # [ 244.001046] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe62240ea2079f1c1bf71f5b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe62240ea2079f1c1bf71f5b: start","job_status":"start","queue_duration_s":56.851618,"scheduling_latency_s":56.85163}2252gitlab # [ 244.001445] gitaly[920]: time="2026-09-03T10:17:03.114Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB8B4JAKC0ZS8FCRAMFKJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.107" grpc.time_ms=6.308 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2253gitlab # [ 244.002028] gitaly[920]: time="2026-09-03T10:17:03.162Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8CDRXSDM2GKWV7BQHVB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.156" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2254gitlab # [ 244.010484] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe62240ea2079f1c1bf71f5b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe62240ea2079f1c1bf71f5b: done: 0.008936 sec","job_status":"done","queue_duration_s":56.851618,"scheduling_latency_s":56.85163,"redis_calls":13,"redis_duration_s":0.0015439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008936,"completed_at":"2026-09-03T10:17:00.113Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2255gitlab # [ 244.055278] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0aa29f94efe3d54f2285cdc0","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0aa29f94efe3d54f2285cdc0: start","job_status":"start","queue_duration_s":56.861349,"scheduling_latency_s":56.861362}2256gitlab # [ 244.056669] gitaly[920]: time="2026-09-03T10:17:03.162Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8CDRXSDM2GKWV7BQHVB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.156" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2257gitlab # [ 244.069401] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0aa29f94efe3d54f2285cdc0","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0aa29f94efe3d54f2285cdc0: done: 0.009023 sec","job_status":"done","queue_duration_s":56.861349,"scheduling_latency_s":56.861362,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009023,"completed_at":"2026-09-03T10:17:00.123Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2258gitlab # [ 244.142335] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ea04ec93436044695854022","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ea04ec93436044695854022: start","job_status":"start","queue_duration_s":56.871172,"scheduling_latency_s":56.871185}2259gitlab # [ 244.142955] gitaly[920]: time="2026-09-03T10:17:03.162Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8CDRXSDM2GKWV7BQHVB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.156" grpc.time_ms=6.254 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2260gitlab # [ 244.143616] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ea04ec93436044695854022","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ea04ec93436044695854022: done: 0.008908 sec","job_status":"done","queue_duration_s":56.871172,"scheduling_latency_s":56.871185,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008908,"completed_at":"2026-09-03T10:17:00.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2261gitlab # [ 244.185585] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f209846b507747b6b22cd84","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f209846b507747b6b22cd84: start","job_status":"start","queue_duration_s":56.880987,"scheduling_latency_s":56.881001}2262gitlab # [ 244.186201] gitaly[920]: time="2026-09-03T10:17:03.227Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB8EE9GX99E4TSZYB5R1K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.220" grpc.time_ms=7.212 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2263gitlab # [ 244.186700] gitaly[920]: time="2026-09-03T10:17:03.283Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8FWNBDFDM277XZAM96H grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.267" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2264gitlab # [ 244.195786] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f209846b507747b6b22cd84","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f209846b507747b6b22cd84: done: 0.008858 sec","job_status":"done","queue_duration_s":56.880987,"scheduling_latency_s":56.881001,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008858,"completed_at":"2026-09-03T10:17:00.142Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2265gitlab # [ 244.245641] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9db63177cf18042f690039d","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9db63177cf18042f690039d: start","job_status":"start","queue_duration_s":56.890744,"scheduling_latency_s":56.890757}2266gitlab # [ 244.246500] gitaly[920]: time="2026-09-03T10:17:03.283Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=12 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8FWNBDFDM277XZAM96H grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.267" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2267gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662'2268gitlab # [ 244.253291] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9db63177cf18042f690039d","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9db63177cf18042f690039d: done: 0.012811 sec","job_status":"done","queue_duration_s":56.890744,"scheduling_latency_s":56.890757,"redis_calls":13,"redis_duration_s":0.0030800000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002334,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012811,"completed_at":"2026-09-03T10:17:00.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2269gitlab # [ 244.302790] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6733527c93316567b1e4cc0e","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6733527c93316567b1e4cc0e: start","job_status":"start","queue_duration_s":56.908832,"scheduling_latency_s":56.908861}2270gitlab # [ 244.303647] gitaly[920]: time="2026-09-03T10:17:03.283Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=12 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8FWNBDFDM277XZAM96H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.267" grpc.time_ms=16.153 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2271gitlab # [ 244.311985] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6733527c93316567b1e4cc0e","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6733527c93316567b1e4cc0e: done: 0.011355 sec","job_status":"done","queue_duration_s":56.908832,"scheduling_latency_s":56.908861,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011355,"completed_at":"2026-09-03T10:17:00.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2272gitlab # [ 244.352055] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7740254f76db8909db10e43","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7740254f76db8909db10e43: start","job_status":"start","queue_duration_s":56.921081,"scheduling_latency_s":56.921094}2273gitlab # [ 244.355762] gitaly[920]: time="2026-09-03T10:17:03.348Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB8JH0QRQ62AB5SG0VXAS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.343" grpc.time_ms=5.042 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2274gitlab # [ 244.404234] gitaly[920]: time="2026-09-03T10:17:03.394Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8KPTN9PT0R374Q5PQDJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.383" pid=920 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2275gitlab # [ 244.404870] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7740254f76db8909db10e43","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7740254f76db8909db10e43: done: 0.009778 sec","job_status":"done","queue_duration_s":56.921081,"scheduling_latency_s":56.921094,"redis_calls":13,"redis_duration_s":0.0014659999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009778,"completed_at":"2026-09-03T10:17:00.183Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2276gitlab # [ 244.411024] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8931b228e6afbe9f6996e74b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8931b228e6afbe9f6996e74b: start","job_status":"start","queue_duration_s":56.931713,"scheduling_latency_s":56.931734}2277gitlab # [ 244.449121] gitaly[920]: time="2026-09-03T10:17:03.394Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8KPTN9PT0R374Q5PQDJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.383" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2278gitlab # [ 244.449800] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8931b228e6afbe9f6996e74b","created_at":"2026-09-03T10:16:03.207Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-e807c30e65004471","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1024514118b560977df21365346911de","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:03.252Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8931b228e6afbe9f6996e74b: done: 0.008944 sec","job_status":"done","queue_duration_s":56.931713,"scheduling_latency_s":56.931734,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008944,"completed_at":"2026-09-03T10:17:00.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279gitlab # [ 244.468745] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.194Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ae739ef73e3fc0c316bb2e41","created_at":"2026-09-03T10:16:04.614Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-c2e0adeefb1c45ae","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:04.619Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ae739ef73e3fc0c316bb2e41: start","job_status":"start","queue_duration_s":55.574335,"scheduling_latency_s":55.574348}2280gitlab # [ 244.479884] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.213Z","message":"No existing merge request to be cleaned up."}2281gitlab # [ 244.481519] gitaly[920]: time="2026-09-03T10:17:03.394Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8KPTN9PT0R374Q5PQDJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:03.383" grpc.time_ms=10.718 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2282gitlab # [ 244.494808] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.222Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ae739ef73e3fc0c316bb2e41","created_at":"2026-09-03T10:16:04.614Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-c2e0adeefb1c45ae","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:04.619Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ae739ef73e3fc0c316bb2e41: done: 0.028349 sec","job_status":"done","queue_duration_s":55.574335,"scheduling_latency_s":55.574348,"redis_calls":14,"redis_duration_s":0.010188,"redis_read_bytes":12,"redis_write_bytes":1678,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004561,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001947,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00368,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":5,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028349,"completed_at":"2026-09-03T10:17:00.222Z","load_balancing_strategy":"primary","db_duration_s":0.003313,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2283gitlab # [ 244.565842] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.223Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7626d68c33f81da631526746","created_at":"2026-09-03T10:16:04.614Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-c2e0adeefb1c45ae","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:04.619Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7626d68c33f81da631526746: start","job_status":"start","queue_duration_s":55.603534,"scheduling_latency_s":55.603558}2284gitlab # [ 244.566724] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.230Z","message":"No existing merge request to be cleaned up."}2285gitlab # [ 244.567299] gitaly[920]: time="2026-09-03T10:17:03.463Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:33.462" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:03.462" grpc.time_ms=0.457 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2286gitlab # [ 244.567922] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.234Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7626d68c33f81da631526746","created_at":"2026-09-03T10:16:04.614Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-c2e0adeefb1c45ae","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:04.619Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7626d68c33f81da631526746: done: 0.011249 sec","job_status":"done","queue_duration_s":55.603534,"scheduling_latency_s":55.603558,"redis_calls":10,"redis_duration_s":0.001197,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000712,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011249,"completed_at":"2026-09-03T10:17:00.234Z","load_balancing_strategy":"primary","db_duration_s":0.001731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2287gitlab # [ 244.623201] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.235Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ded9665af0be3a609509ce7e","created_at":"2026-09-03T10:16:04.614Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-c2e0adeefb1c45ae","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:04.619Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ded9665af0be3a609509ce7e: start","job_status":"start","queue_duration_s":55.615588,"scheduling_latency_s":55.615602}2288gitlab # [ 244.623874] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.241Z","message":"No existing merge request to be cleaned up."}2289gitlab # [ 244.624554] gitaly[920]: time="2026-09-03T10:17:03.474Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-03T10:17:33.468" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:03.468" grpc.time_ms=5.472 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2290gitlab # [ 244.625189] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.246Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ded9665af0be3a609509ce7e","created_at":"2026-09-03T10:16:04.614Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-c2e0adeefb1c45ae","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:04.619Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ded9665af0be3a609509ce7e: done: 0.010722 sec","job_status":"done","queue_duration_s":55.615588,"scheduling_latency_s":55.615602,"redis_calls":10,"redis_duration_s":0.0012079999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000712,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010722,"completed_at":"2026-09-03T10:17:00.246Z","load_balancing_strategy":"primary","db_duration_s":0.001511,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2291gitlab # [ 244.647523] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.247Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"35686eead4b8f6d88783231a","created_at":"2026-09-03T10:16:04.614Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-c2e0adeefb1c45ae","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:04.619Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-35686eead4b8f6d88783231a: start","job_status":"start","queue_duration_s":55.627136,"scheduling_latency_s":55.627149}2292gitlab # [ 244.658733] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.254Z","message":"No existing merge request to be cleaned up."}2293gitlab # [ 244.659992] gitaly[920]: time="2026-09-03T10:17:03.484Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:13.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:03.478" grpc.time_ms=5.164 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2294gitlab # [ 244.672180] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.262Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"35686eead4b8f6d88783231a","created_at":"2026-09-03T10:16:04.614Z","trace_propagation_headers":{"sentry-trace":"ffa96a852be244ffb2b039b7c91e62c6-c2e0adeefb1c45ae","baggage":"sentry-trace_id=ffa96a852be244ffb2b039b7c91e62c6,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"882aea428a9242fa2ad37e59634f9e98","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:04.619Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-35686eead4b8f6d88783231a: done: 0.01547 sec","job_status":"done","queue_duration_s":55.627136,"scheduling_latency_s":55.627149,"redis_calls":10,"redis_duration_s":0.004976,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00355,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01547,"completed_at":"2026-09-03T10:17:00.262Z","load_balancing_strategy":"primary","db_duration_s":0.001681,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2295gitlab # [ 244.728174] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.265Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"083781d1f4b6a32072cbcdeb","created_at":"2026-09-03T10:16:08.287Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.332Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-083781d1f4b6a32072cbcdeb: start","job_status":"start","queue_duration_s":51.932415,"scheduling_latency_s":51.93243}2296gitlab # [ 244.728810] gitaly[920]: time="2026-09-03T10:17:03.500Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-03T10:17:13.030" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:03.500" grpc.time_ms=0.143 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2297gitlab # [ 244.729477] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.362Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"083781d1f4b6a32072cbcdeb","created_at":"2026-09-03T10:16:08.287Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.332Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-083781d1f4b6a32072cbcdeb: done: 0.0974 sec","job_status":"done","queue_duration_s":51.932415,"scheduling_latency_s":51.93243,"redis_calls":13,"redis_duration_s":0.0048130000000000004,"redis_read_bytes":138,"redis_write_bytes":1593,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001273,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":428,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002125,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001415,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.066648694,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.0974,"completed_at":"2026-09-03T10:17:00.362Z","load_balancing_strategy":"primary","db_duration_s":0.035949,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2298gitlab # [ 244.769592] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.364Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"e5d8b7f5f2fc091a871038f7","created_at":"2026-09-03T10:16:08.348Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.356Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-e5d8b7f5f2fc091a871038f7: start","job_status":"start","queue_duration_s":52.008566,"scheduling_latency_s":52.008599}2299gitlab # [ 244.808852] gitaly[920]: time="2026-09-03T10:17:03.508Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-03T10:17:13.033" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:03.503" grpc.time_ms=4.41 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2300gitlab # [ 244.809628] gitaly[920]: time="2026-09-03T10:17:03.533Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:03.533" grpc.time_ms=0.209 pid=920 span.kind=server system=grpc2301gitlab # [ 244.810134] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.428Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"e5d8b7f5f2fc091a871038f7","created_at":"2026-09-03T10:16:08.348Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.356Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-e5d8b7f5f2fc091a871038f7: done: 0.063908 sec","job_status":"done","queue_duration_s":52.008566,"scheduling_latency_s":52.008599,"redis_calls":14,"redis_duration_s":0.001915,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000387,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000627,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.047427536,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.063908,"completed_at":"2026-09-03T10:17:00.428Z","load_balancing_strategy":"primary","db_duration_s":0.012655,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2302gitlab # [ 244.849504] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.429Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"b335b237fc0ecf99223a78f3","created_at":"2026-09-03T10:16:08.901Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.906Z","job_size_bytes":5,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-b335b237fc0ecf99223a78f3: start","job_status":"start","queue_duration_s":51.523673,"scheduling_latency_s":51.523688}2303gitlab # [ 244.889206] gitaly[920]: time="2026-09-03T10:17:03.554Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:03.553" grpc.time_ms=0.404 pid=920 span.kind=server system=grpc2304gitlab # [ 244.889810] gitaly[920]: time="2026-09-03T10:17:03.557Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:03.557" grpc.time_ms=0.161 pid=920 span.kind=server system=grpc2305gitlab # [ 244.890525] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.528Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"b335b237fc0ecf99223a78f3","created_at":"2026-09-03T10:16:08.901Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.906Z","job_size_bytes":5,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-b335b237fc0ecf99223a78f3: done: 0.098332 sec","job_status":"done","queue_duration_s":51.523673,"scheduling_latency_s":51.523688,"redis_calls":12,"redis_duration_s":0.001742,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001093,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212038,"duration_s":0.098332,"completed_at":"2026-09-03T10:17:00.528Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013845,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2306gitlab # [ 244.933906] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.529Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"00ed327831817dc29f256e19","created_at":"2026-09-03T10:16:08.926Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.929Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-00ed327831817dc29f256e19: start","job_status":"start","queue_duration_s":51.59956,"scheduling_latency_s":51.599575}2307gitlab # [ 244.934629] gitaly[920]: time="2026-09-03T10:17:03.563Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 diskcache=56e6f453-15d6-4ec7-bccb-c310ca3e9008 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:17:13.039" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:03.509" pid=920 ref=refs/keep-around/eba459b7c8f87c78f7a7240ca13a81922b9f7662 remote_ip="2001:db8:1::1" revision=eba459b7c8f87c78f7a7240ca13a81922b9f7662 span.kind=server system=grpc username=root2308gitlab # [ 244.936837] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.647Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"00ed327831817dc29f256e19","created_at":"2026-09-03T10:16:08.926Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.929Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-00ed327831817dc29f256e19: done: 0.117764 sec","job_status":"done","queue_duration_s":51.59956,"scheduling_latency_s":51.599575,"redis_calls":11,"redis_duration_s":0.001497,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000368,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.096,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.096,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000204495,"duration_s":0.117764,"completed_at":"2026-09-03T10:17:00.647Z","load_balancing_strategy":"primary","db_duration_s":0.036495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2309gitlab # [ 244.979297] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.648Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c28051e4705cb80e7304bb07","created_at":"2026-09-03T10:16:08.931Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.934Z","job_size_bytes":89,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c28051e4705cb80e7304bb07: start","job_status":"start","queue_duration_s":51.713529,"scheduling_latency_s":51.713543}2310gitlab # [ 245.026335] gitaly[920]: time="2026-09-03T10:17:03.563Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=45 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5466 command.oublock=8 command.real_time_ms=47 command.spawn_token_fork_ms=0 command.system_time_ms=33 command.user_time_ms=12 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:17:13.039" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:17:03.509" grpc.time_ms=53.781 pid=920 ref=refs/keep-around/eba459b7c8f87c78f7a7240ca13a81922b9f7662 remote_ip="2001:db8:1::1" revision=eba459b7c8f87c78f7a7240ca13a81922b9f7662 span.kind=server system=grpc username=root2311gitlab # [ 245.026984] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.669Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c28051e4705cb80e7304bb07","created_at":"2026-09-03T10:16:08.931Z","trace_propagation_headers":{"sentry-trace":"10a4544b8ff342259316d96717e354f0-c3f50f56b0214cd4","baggage":"sentry-trace_id=10a4544b8ff342259316d96717e354f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9HN0FXRW6W95WN87MVGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:08.934Z","job_size_bytes":89,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c28051e4705cb80e7304bb07: done: 0.021442 sec","job_status":"done","queue_duration_s":51.713529,"scheduling_latency_s":51.713543,"redis_calls":11,"redis_duration_s":0.001488,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000353,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000381,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000191086,"duration_s":0.021442,"completed_at":"2026-09-03T10:17:00.669Z","load_balancing_strategy":"primary","db_duration_s":0.00255,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2312gitlab # [ 245.043758] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.670Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 eedce16a998e57e3e500c437ccc385d7d5ac4a49 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"8e62e6eb8e33c5e027077f03","created_at":"2026-09-03T10:16:10.196Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7d70515b9ff446e5","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:10.214Z","job_size_bytes":185,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-8e62e6eb8e33c5e027077f03: start","job_status":"start","queue_duration_s":50.455604,"scheduling_latency_s":50.455618}2313gitlab # [ 245.092631] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:00.679Z","message":"changes: \"0000000000000000000000000000000000000000 eedce16a998e57e3e500c437ccc385d7d5ac4a49 refs/heads/master\\n\""}2314gitlab # [ 245.093259] gitaly[920]: time="2026-09-03T10:17:03.567Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:33.566" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:03.566" grpc.time_ms=0.251 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2315gitlab # [ 245.094026] gitaly[920]: time="2026-09-03T10:17:03.574Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:03.574" grpc.time_ms=0.192 pid=920 span.kind=server system=grpc2316gitlab # [ 245.094644] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.278Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 eedce16a998e57e3e500c437ccc385d7d5ac4a49 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"8e62e6eb8e33c5e027077f03","created_at":"2026-09-03T10:16:10.196Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7d70515b9ff446e5","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:10.214Z","job_size_bytes":185,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-8e62e6eb8e33c5e027077f03: done: 0.608102 sec","job_status":"done","queue_duration_s":50.455604,"scheduling_latency_s":50.455618,"gitaly_calls":13,"gitaly_duration_s":0.121367,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.02663,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.002263,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.003452,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.000938,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002784,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.009098,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.008095,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.154,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.167,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.429618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055845,"exclusive_lock_hold_duration_s":0.022849832,"duration_s":0.608102,"completed_at":"2026-09-03T10:17:01.278Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050519,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2317gitlab # [ 245.146003] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.279Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","eedce16a998e57e3e500c437ccc385d7d5ac4a49 eba459b7c8f87c78f7a7240ca13a81922b9f7662 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"69a3a97c72f0041597c9277a","created_at":"2026-09-03T10:16:13.360Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-ff6c9f2589cc49f3","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:13.361Z","job_size_bytes":185,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-69a3a97c72f0041597c9277a: start","job_status":"start","queue_duration_s":47.918561,"scheduling_latency_s":47.918578}2318gitlab # [ 245.146815] gitaly[920]: time="2026-09-03T10:17:03.596Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:03.595" grpc.time_ms=0.794 pid=920 span.kind=server system=grpc2319gitlab # [ 245.149281] gitaly[920]: time="2026-09-03T10:17:03.609Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X diskcache=84c6fccb-7515-4dce-be0f-7110074825fe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:33.530" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:03.530" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2320gitlab # [ 245.158608] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.287Z","message":"changes: \"eedce16a998e57e3e500c437ccc385d7d5ac4a49 eba459b7c8f87c78f7a7240ca13a81922b9f7662 refs/heads/master\\n\""}2321gitlab # [ 245.160571] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.314Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ad71e7729524695d3830cfd8","created_at":"2026-09-03T10:17:01.313Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-0bfa3b571e4d4c0e","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"c27d0e179b5ac60c3051d5bf","job_size_bytes":46,"pid":2556,"job_status":"deduplicated","message":"ProjectCacheWorker JID-ad71e7729524695d3830cfd8: deduplicated: until executing","deduplication.type":"until executing"}2322gitlab # [ 245.211719] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.417Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","eedce16a998e57e3e500c437ccc385d7d5ac4a49 eba459b7c8f87c78f7a7240ca13a81922b9f7662 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"69a3a97c72f0041597c9277a","created_at":"2026-09-03T10:16:13.360Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-ff6c9f2589cc49f3","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:16:13.361Z","job_size_bytes":185,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-69a3a97c72f0041597c9277a: done: 0.137434 sec","job_status":"done","queue_duration_s":47.918561,"scheduling_latency_s":47.918578,"gitaly_calls":7,"gitaly_duration_s":0.039408,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.012563000000000001,"redis_read_bytes":500,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.001324,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000532,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002308,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.004253,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.004146,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000344178,"exclusive_lock_hold_duration_s":0.011423239,"duration_s":0.137434,"completed_at":"2026-09-03T10:17:01.417Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016032,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2323gitlab # [ 245.217722] gitaly[920]: time="2026-09-03T10:17:03.609Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7381 command.oublock=0 command.real_time_ms=72 command.spawn_token_fork_ms=2 command.system_time_ms=43 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:33.530" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:03.530" grpc.time_ms=79.006 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2324gitlab # [ 245.229721] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.418Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"4671dcaff989179da52e7552","created_at":"2026-09-03T10:16:14.434Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-d76607b10f114d3c","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:14.439Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-4671dcaff989179da52e7552: start","job_status":"start","queue_duration_s":46.979255,"scheduling_latency_s":46.979271}2325gitlab # [ 245.244720] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.465Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"4671dcaff989179da52e7552","created_at":"2026-09-03T10:16:14.434Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-d76607b10f114d3c","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:14.439Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-4671dcaff989179da52e7552: done: 0.047191 sec","job_status":"done","queue_duration_s":46.979255,"scheduling_latency_s":46.979271,"redis_calls":16,"redis_duration_s":0.002107,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001092,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210362,"duration_s":0.047191,"completed_at":"2026-09-03T10:17:01.465Z","load_balancing_strategy":"primary","db_duration_s":0.009949,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2326gitlab # [ 245.292733] gitaly[920]: time="2026-09-03T10:17:03.646Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:17:33.645" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:03.645" grpc.time_ms=0.961 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2327gitlab # [ 245.294951] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.466Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"63e5146af8ac55a3e3ad7b58","created_at":"2026-09-03T10:16:14.439Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-d76607b10f114d3c","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:14.442Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-63e5146af8ac55a3e3ad7b58: start","job_status":"start","queue_duration_s":47.023606,"scheduling_latency_s":47.023619}2328gitlab # [ 245.334084] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.523Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"63e5146af8ac55a3e3ad7b58","created_at":"2026-09-03T10:16:14.439Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-d76607b10f114d3c","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:14.442Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-63e5146af8ac55a3e3ad7b58: done: 0.056785 sec","job_status":"done","queue_duration_s":47.023606,"scheduling_latency_s":47.023619,"redis_calls":21,"redis_duration_s":0.002744,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.001147,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000243606,"duration_s":0.056785,"completed_at":"2026-09-03T10:17:01.523Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2329gitlab # [ 245.381152] gitaly[920]: time="2026-09-03T10:17:03.668Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KC9RA7SASYBQ7BHQKAM95X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:33.667" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:03.667" grpc.time_ms=0.427 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2330gitlab # [ 245.381801] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.524Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5d28c7478b1b60672368b9b7","created_at":"2026-09-03T10:16:14.525Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-d76607b10f114d3c","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:14.531Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5d28c7478b1b60672368b9b7: start","job_status":"start","queue_duration_s":46.993119,"scheduling_latency_s":46.993135}2331gitlab # [ 245.382414] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.761Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5d28c7478b1b60672368b9b7","created_at":"2026-09-03T10:16:14.525Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-d76607b10f114d3c","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:14.531Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2556,"job_status":"rescheduled","message":"PipelineProcessWorker JID-5d28c7478b1b60672368b9b7: rescheduled"}2332gitlab # [ 245.424243] gitaly[920]: time="2026-09-03T10:17:03.884Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:17:33.883" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:03.883" grpc.time_ms=0.579 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2333gitlab # [ 245.424922] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.767Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5d28c7478b1b60672368b9b7","created_at":"2026-09-03T10:16:14.525Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-d76607b10f114d3c","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:16:14.531Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5d28c7478b1b60672368b9b7: done: 0.24311 sec","job_status":"done","queue_duration_s":46.993119,"scheduling_latency_s":46.993135,"gitaly_calls":3,"gitaly_duration_s":0.073017,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014803,"redis_read_bytes":462,"redis_write_bytes":29297,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000699,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000842,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.005357,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.002198,"redis_queues_read_bytes":39,"redis_queues_write_bytes":15584,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.003161,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000303,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000327,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.129627,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.210201577,"duration_s":0.24311,"completed_at":"2026-09-03T10:17:01.767Z","load_balancing_strategy":"primary","db_duration_s":0.034392,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2334gitlab # [ 245.465203] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.768Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c27d0e179b5ac60c3051d5bf","created_at":"2026-09-03T10:17:00.756Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:00.768Z","job_size_bytes":46,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c27d0e179b5ac60c3051d5bf: start","job_status":"start","queue_duration_s":1.000399,"scheduling_latency_s":1.000414}2335gitlab # [ 245.465879] gitaly[920]: time="2026-09-03T10:17:03.907Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCB8P09GWG8TDVRMRHM6Z2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:33.607" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:03.907" grpc.time_ms=0.433 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2336gitlab # [ 245.474115] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.823Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c27d0e179b5ac60c3051d5bf","created_at":"2026-09-03T10:17:00.756Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:00.768Z","job_size_bytes":46,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-c27d0e179b5ac60c3051d5bf: done: 0.055034 sec","job_status":"done","queue_duration_s":1.000399,"scheduling_latency_s":1.000414,"gitaly_calls":2,"gitaly_duration_s":0.010034,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.003987,"redis_read_bytes":69,"redis_write_bytes":4883,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.00075,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.00036,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00157,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000375,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000198908,"duration_s":0.055034,"completed_at":"2026-09-03T10:17:01.823Z","load_balancing_strategy":"primary","db_duration_s":0.007236,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2337gitlab # [ 245.536702] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.824Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0e23cd20e16f88c3bae2e2ab","created_at":"2026-09-03T10:17:00.776Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:00.778Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0e23cd20e16f88c3bae2e2ab: start","job_status":"start","queue_duration_s":1.045805,"scheduling_latency_s":1.045819}2338gitlab # [ 245.537569] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.885Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0e23cd20e16f88c3bae2e2ab","created_at":"2026-09-03T10:17:00.776Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:00.778Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0e23cd20e16f88c3bae2e2ab: done: 0.060709 sec","job_status":"done","queue_duration_s":1.045805,"scheduling_latency_s":1.045819,"gitaly_calls":1,"gitaly_duration_s":0.020591,"redis_calls":15,"redis_duration_s":0.001983,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000244,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000335,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000654,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.042641451999999996,"duration_s":0.060709,"completed_at":"2026-09-03T10:17:01.885Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2339gitlab # [ 245.582790] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.886Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8cc35155041fcaf0f63706f2","created_at":"2026-09-03T10:17:00.843Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:00.844Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8cc35155041fcaf0f63706f2: start","job_status":"start","queue_duration_s":1.041387,"scheduling_latency_s":1.041402}2340gitlab # [ 245.586248] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.915Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8cc35155041fcaf0f63706f2","created_at":"2026-09-03T10:17:00.843Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:00.844Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8cc35155041fcaf0f63706f2: done: 0.029565 sec","job_status":"done","queue_duration_s":1.041387,"scheduling_latency_s":1.041402,"gitaly_calls":1,"gitaly_duration_s":0.00684,"redis_calls":16,"redis_duration_s":0.0023510000000000002,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000187,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001038,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029565,"completed_at":"2026-09-03T10:17:01.915Z","load_balancing_strategy":"primary","db_duration_s":0.001999,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2341gitlab # [ 245.654239] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:01.916Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"d864fe7c4e552a756bedf2ad","created_at":"2026-09-03T10:17:01.149Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:eacfb2f8c8baf3a4d6df31181a3e84033ddf6452e9e79abd67f5353ce5ec37e9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.152Z","job_size_bytes":467,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-d864fe7c4e552a756bedf2ad: start","job_status":"start","queue_duration_s":0.764323,"scheduling_latency_s":0.764342}2342gitlab # [ 245.654939] sidekiq-cluster[2556]: {"severity":"WARN","time":"2026-09-03T10:17:02.491Z","class":"CreatePipelineWorker","correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"eedce16a998e57e3e500c437ccc385d7d5ac4a49","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"eedce16a998e57e3e500c437ccc385d7d5ac4a49","ref":"refs/heads/master"},"retry":0}2343gitlab # [ 245.655594] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:02.496Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"d864fe7c4e552a756bedf2ad","created_at":"2026-09-03T10:17:01.149Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:eacfb2f8c8baf3a4d6df31181a3e84033ddf6452e9e79abd67f5353ce5ec37e9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.152Z","job_size_bytes":467,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-d864fe7c4e552a756bedf2ad: done: 0.579317 sec","job_status":"done","queue_duration_s":0.764323,"scheduling_latency_s":0.764342,"gitaly_calls":7,"gitaly_duration_s":0.041571,"redis_calls":51,"redis_duration_s":0.005802,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.000514,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.002504,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001457,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000597,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.504671,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000192762,"duration_s":0.579317,"completed_at":"2026-09-03T10:17:02.496Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025207,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2344gitlab # [ 245.697375] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:02.497Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"8a77bb57-00b5-4c2b-877d-2ee7222a7966\"}"],"class":"WebHookWorker","jid":"06e6874a852a4c3815afd29f","created_at":"2026-09-03T10:17:01.237Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f16c72f01be03aa98beeae8ba7a35ac7fa8c88b3e403a0e9116ae800b03c6008","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.240Z","job_size_bytes":1751,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-06e6874a852a4c3815afd29f: start","job_status":"start","queue_duration_s":1.256498,"scheduling_latency_s":1.256513}2345gitlab # [ 245.707099] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:02.590Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"8a77bb57-00b5-4c2b-877d-2ee7222a7966\"}"],"class":"WebHookWorker","jid":"06e6874a852a4c3815afd29f","created_at":"2026-09-03T10:17:01.237Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-7c91bf30e15045e3","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f16c72f01be03aa98beeae8ba7a35ac7fa8c88b3e403a0e9116ae800b03c6008","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.240Z","job_size_bytes":1751,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-06e6874a852a4c3815afd29f: done: 0.093114 sec","job_status":"done","queue_duration_s":1.256498,"scheduling_latency_s":1.256513,"redis_calls":19,"redis_duration_s":0.002852,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000631,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000213,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001524,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.028726556000009396,"cpu_s":0.054693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205892,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.093114,"completed_at":"2026-09-03T10:17:02.590Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00188,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2346gitlab # [ 245.778339] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:02.591Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a904b13b58f4e7ac4961bda3","created_at":"2026-09-03T10:17:01.317Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-0bfa3b571e4d4c0e","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"0e23cd20e16f88c3bae2e2ab","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.319Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a904b13b58f4e7ac4961bda3: start","job_status":"start","queue_duration_s":1.272406,"scheduling_latency_s":1.272421}2347gitlab # [ 245.778971] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:02.630Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a904b13b58f4e7ac4961bda3","created_at":"2026-09-03T10:17:01.317Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-0bfa3b571e4d4c0e","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"0e23cd20e16f88c3bae2e2ab","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.319Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a904b13b58f4e7ac4961bda3: done: 0.039492 sec","job_status":"done","queue_duration_s":1.272406,"scheduling_latency_s":1.272421,"gitaly_calls":1,"gitaly_duration_s":0.014057,"redis_calls":12,"redis_duration_s":0.0026490000000000003,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001072,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000644,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.024805946,"duration_s":0.039492,"completed_at":"2026-09-03T10:17:02.630Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006406,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2348gitlab # [ 245.822714] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:02.632Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"aa15ecfa2573a1cb296e6750","created_at":"2026-09-03T10:17:01.386Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-0bfa3b571e4d4c0e","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:75bd99b8370d76297d4804a424b23035412f6aee37c3619e5b840804c156036a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.387Z","job_size_bytes":467,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-aa15ecfa2573a1cb296e6750: start","job_status":"start","queue_duration_s":1.244969,"scheduling_latency_s":1.244984}2349gitlab # [ 245.823538] sidekiq-cluster[2556]: {"severity":"WARN","time":"2026-09-03T10:17:03.075Z","class":"CreatePipelineWorker","correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"eedce16a998e57e3e500c437ccc385d7d5ac4a49","after":"eba459b7c8f87c78f7a7240ca13a81922b9f7662","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"eba459b7c8f87c78f7a7240ca13a81922b9f7662","ref":"refs/heads/master"},"retry":0}2350gitlab # [ 245.827703] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"aa15ecfa2573a1cb296e6750","created_at":"2026-09-03T10:17:01.386Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-0bfa3b571e4d4c0e","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:75bd99b8370d76297d4804a424b23035412f6aee37c3619e5b840804c156036a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.387Z","job_size_bytes":467,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-aa15ecfa2573a1cb296e6750: done: 0.449244 sec","job_status":"done","queue_duration_s":1.244969,"scheduling_latency_s":1.244984,"gitaly_calls":7,"gitaly_duration_s":0.046423,"redis_calls":33,"redis_duration_s":0.006024,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000444,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002285,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001503,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.381736,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.449244,"completed_at":"2026-09-03T10:17:03.082Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012858,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2351gitlab # [ 245.902824] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.085Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"a87774a2-151d-4c7b-9f1c-7a10acb13773\"}"],"class":"WebHookWorker","jid":"d8d2993b38364f9222a22b86","created_at":"2026-09-03T10:17:01.396Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-0bfa3b571e4d4c0e","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:30687d38ffe85ac081356ecde4e8b1230e49ab359fa168164482f5781aa17715","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.397Z","job_size_bytes":1749,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d8d2993b38364f9222a22b86: start","job_status":"start","queue_duration_s":1.687703,"scheduling_latency_s":1.687723}2352gitlab # [ 245.903673] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.121Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"a87774a2-151d-4c7b-9f1c-7a10acb13773\"}"],"class":"WebHookWorker","jid":"d8d2993b38364f9222a22b86","created_at":"2026-09-03T10:17:01.396Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-0bfa3b571e4d4c0e","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:30687d38ffe85ac081356ecde4e8b1230e49ab359fa168164482f5781aa17715","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.397Z","job_size_bytes":1749,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d8d2993b38364f9222a22b86: done: 0.035712 sec","job_status":"done","queue_duration_s":1.687703,"scheduling_latency_s":1.687723,"redis_calls":14,"redis_duration_s":0.004406,"redis_read_bytes":26,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000224,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002904,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.006396064000000479,"cpu_s":0.015875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.035712,"completed_at":"2026-09-03T10:17:03.121Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001172,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2353gitlab # [ 245.947305] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9a4df2e8d6956a5aae68114e","created_at":"2026-09-03T10:17:01.589Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.592Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9a4df2e8d6956a5aae68114e: start","job_status":"start","queue_duration_s":1.529944,"scheduling_latency_s":1.529963}2354gitlab # [ 245.947938] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.184Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9a4df2e8d6956a5aae68114e","created_at":"2026-09-03T10:17:01.589Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.592Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9a4df2e8d6956a5aae68114e: done: 0.06228 sec","job_status":"done","queue_duration_s":1.529944,"scheduling_latency_s":1.529963,"redis_calls":10,"redis_duration_s":0.001744,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001082,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000403683,"duration_s":0.06228,"completed_at":"2026-09-03T10:17:03.184Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015624,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2355gitlab # [ 246.006197] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.187Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a9d51e3aba8f424e5dfcfb99","created_at":"2026-09-03T10:17:01.669Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.672Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a9d51e3aba8f424e5dfcfb99: start","job_status":"start","queue_duration_s":1.514596,"scheduling_latency_s":1.514625}2356gitlab # [ 246.006960] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.216Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a9d51e3aba8f424e5dfcfb99","created_at":"2026-09-03T10:17:01.669Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.672Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a9d51e3aba8f424e5dfcfb99: done: 0.029213 sec","job_status":"done","queue_duration_s":1.514596,"scheduling_latency_s":1.514625,"redis_calls":10,"redis_duration_s":0.002265,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001036,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000245003,"duration_s":0.029213,"completed_at":"2026-09-03T10:17:03.216Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006804,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2357gitlab # [ 246.035771] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.217Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"95f09bad348204f236522159","created_at":"2026-09-03T10:17:01.680Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.685Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-95f09bad348204f236522159: start","job_status":"start","queue_duration_s":1.531541,"scheduling_latency_s":1.531555}2358gitlab # [ 246.087162] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"95f09bad348204f236522159","created_at":"2026-09-03T10:17:01.680Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.685Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-95f09bad348204f236522159: done: 0.021476 sec","job_status":"done","queue_duration_s":1.531541,"scheduling_latency_s":1.531555,"redis_calls":10,"redis_duration_s":0.001362,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000230756,"duration_s":0.021476,"completed_at":"2026-09-03T10:17:03.238Z","load_balancing_strategy":"primary","db_duration_s":0.003285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2359gitlab # [ 246.088260] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.239Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"9b273133ee6862a36c30a1a3","created_at":"2026-09-03T10:17:01.694Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.696Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9b273133ee6862a36c30a1a3: start","job_status":"start","queue_duration_s":1.543046,"scheduling_latency_s":1.54306}2360gitlab # [ 246.102981] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.273Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"9b273133ee6862a36c30a1a3","created_at":"2026-09-03T10:17:01.694Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.696Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9b273133ee6862a36c30a1a3: done: 0.033775 sec","job_status":"done","queue_duration_s":1.543046,"scheduling_latency_s":1.54306,"redis_calls":13,"redis_duration_s":0.002957,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001248,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000342222,"duration_s":0.033775,"completed_at":"2026-09-03T10:17:03.273Z","load_balancing_strategy":"primary","db_duration_s":0.008918,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2361gitlab # [ 246.165142] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.274Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"02935b3ca1d1bdd6afe1f233","created_at":"2026-09-03T10:17:01.697Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.699Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-02935b3ca1d1bdd6afe1f233: start","job_status":"start","queue_duration_s":1.575534,"scheduling_latency_s":1.575549}2362gitlab # [ 246.165904] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.296Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"02935b3ca1d1bdd6afe1f233","created_at":"2026-09-03T10:17:01.697Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.699Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-02935b3ca1d1bdd6afe1f233: done: 0.021731 sec","job_status":"done","queue_duration_s":1.575534,"scheduling_latency_s":1.575549,"redis_calls":10,"redis_duration_s":0.0013059999999999999,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200584,"duration_s":0.021731,"completed_at":"2026-09-03T10:17:03.296Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003492,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2363gitlab # [ 246.222857] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.297Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6cab050e973f513266b98d90","created_at":"2026-09-03T10:17:01.699Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.702Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6cab050e973f513266b98d90: start","job_status":"start","queue_duration_s":1.595546,"scheduling_latency_s":1.59556}2364gitlab # [ 246.223667] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.317Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6cab050e973f513266b98d90","created_at":"2026-09-03T10:17:01.699Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.702Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6cab050e973f513266b98d90: done: 0.020265 sec","job_status":"done","queue_duration_s":1.595546,"scheduling_latency_s":1.59556,"redis_calls":11,"redis_duration_s":0.003538,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002784,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000512,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000640584,"duration_s":0.020265,"completed_at":"2026-09-03T10:17:03.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2365gitlab # [ 246.249784] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.318Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b1d5611a75b016c4cc1df242","created_at":"2026-09-03T10:17:01.703Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.706Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b1d5611a75b016c4cc1df242: start","job_status":"start","queue_duration_s":1.612659,"scheduling_latency_s":1.612672}2366gitlab # [ 246.265170] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.338Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b1d5611a75b016c4cc1df242","created_at":"2026-09-03T10:17:01.703Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.706Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b1d5611a75b016c4cc1df242: done: 0.019756 sec","job_status":"done","queue_duration_s":1.612659,"scheduling_latency_s":1.612672,"redis_calls":10,"redis_duration_s":0.001314,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226845,"duration_s":0.019756,"completed_at":"2026-09-03T10:17:03.338Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003843,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2367gitlab # [ 246.341683] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.339Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c8b9b326ef8c5d9bbf94dfe3","created_at":"2026-09-03T10:17:01.706Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.708Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c8b9b326ef8c5d9bbf94dfe3: start","job_status":"start","queue_duration_s":1.630935,"scheduling_latency_s":1.630948}2368gitlab # [ 246.342624] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.366Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c8b9b326ef8c5d9bbf94dfe3","created_at":"2026-09-03T10:17:01.706Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.708Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c8b9b326ef8c5d9bbf94dfe3: done: 0.026352 sec","job_status":"done","queue_duration_s":1.630935,"scheduling_latency_s":1.630948,"redis_calls":10,"redis_duration_s":0.001513,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00036,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000204774,"duration_s":0.026352,"completed_at":"2026-09-03T10:17:03.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005114,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2369gitlab # [ 246.386257] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.367Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7769236896fbaede0495380c","created_at":"2026-09-03T10:17:01.712Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.714Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7769236896fbaede0495380c: start","job_status":"start","queue_duration_s":1.652509,"scheduling_latency_s":1.652523}2370gitlab # [ 246.386863] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.395Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7769236896fbaede0495380c","created_at":"2026-09-03T10:17:01.712Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.714Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7769236896fbaede0495380c: done: 0.028013 sec","job_status":"done","queue_duration_s":1.652509,"scheduling_latency_s":1.652523,"redis_calls":10,"redis_duration_s":0.0035789999999999997,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000545,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001584,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000304508,"duration_s":0.028013,"completed_at":"2026-09-03T10:17:03.395Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006915,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2371gitlab # [ 246.425082] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.396Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"82194bf21557002e62150100","created_at":"2026-09-03T10:17:01.714Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.717Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-82194bf21557002e62150100: start","job_status":"start","queue_duration_s":1.679151,"scheduling_latency_s":1.679165}2372gitlab # [ 246.425813] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.411Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"82194bf21557002e62150100","created_at":"2026-09-03T10:17:01.714Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.717Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-82194bf21557002e62150100: done: 0.015662 sec","job_status":"done","queue_duration_s":1.679151,"scheduling_latency_s":1.679165,"redis_calls":10,"redis_duration_s":0.001645,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000374,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000238019,"duration_s":0.015662,"completed_at":"2026-09-03T10:17:03.411Z","load_balancing_strategy":"primary","db_duration_s":0.002537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2373gitlab # [ 246.466551] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.413Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e73e1cd481e4a7245a140974","created_at":"2026-09-03T10:17:01.717Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.719Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e73e1cd481e4a7245a140974: start","job_status":"start","queue_duration_s":1.693385,"scheduling_latency_s":1.693398}2374gitlab # [ 246.476707] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.435Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e73e1cd481e4a7245a140974","created_at":"2026-09-03T10:17:01.717Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.719Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e73e1cd481e4a7245a140974: done: 0.022499 sec","job_status":"done","queue_duration_s":1.693385,"scheduling_latency_s":1.693398,"redis_calls":10,"redis_duration_s":0.002187,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000206451,"duration_s":0.022499,"completed_at":"2026-09-03T10:17:03.435Z","load_balancing_strategy":"primary","db_duration_s":0.003946,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2375gitlab # [ 246.515171] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.436Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"0adff3f3a4d56a34170f6c6c","created_at":"2026-09-03T10:17:01.723Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.725Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-0adff3f3a4d56a34170f6c6c: start","job_status":"start","queue_duration_s":1.711235,"scheduling_latency_s":1.711247}2376gitlab # [ 246.529694] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.480Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"0adff3f3a4d56a34170f6c6c","created_at":"2026-09-03T10:17:01.723Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.725Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-0adff3f3a4d56a34170f6c6c: done: 0.043732 sec","job_status":"done","queue_duration_s":1.711235,"scheduling_latency_s":1.711247,"redis_calls":14,"redis_duration_s":0.002348,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000786,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000443352,"duration_s":0.043732,"completed_at":"2026-09-03T10:17:03.480Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010787,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2377gitlab # [ 246.569704] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.481Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ce98327c330c3ed48cf877e9","created_at":"2026-09-03T10:17:01.762Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.763Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ce98327c330c3ed48cf877e9: start","job_status":"start","queue_duration_s":1.717861,"scheduling_latency_s":1.717877}2378gitlab # [ 246.617223] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.677Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ce98327c330c3ed48cf877e9","created_at":"2026-09-03T10:17:01.762Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.763Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2556,"job_status":"rescheduled","message":"PipelineProcessWorker JID-ce98327c330c3ed48cf877e9: rescheduled"}2379gitlab # [ 246.617842] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.682Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ce98327c330c3ed48cf877e9","created_at":"2026-09-03T10:17:01.762Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5ea01373dfdf42d9","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.763Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ce98327c330c3ed48cf877e9: done: 0.201429 sec","job_status":"done","queue_duration_s":1.717861,"scheduling_latency_s":1.717877,"gitaly_calls":3,"gitaly_duration_s":0.085526,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.011143,"redis_read_bytes":152,"redis_write_bytes":18018,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000855,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000572,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002053,"redis_queues_read_bytes":30,"redis_queues_write_bytes":12317,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.003124,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000165,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000744,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002775,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080986,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.171306967,"duration_s":0.201429,"completed_at":"2026-09-03T10:17:03.682Z","load_balancing_strategy":"primary","db_duration_s":0.027667,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2380gitlab # [ 246.664118] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.683Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c047a49c9bcb410a9c458d0f","created_at":"2026-09-03T10:17:01.813Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-4c8cf4385e024abc","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.816Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c047a49c9bcb410a9c458d0f: start","job_status":"start","queue_duration_s":1.867166,"scheduling_latency_s":1.867185}2381gitlab # [ 246.664767] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.714Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c047a49c9bcb410a9c458d0f","created_at":"2026-09-03T10:17:01.813Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-4c8cf4385e024abc","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:01.816Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c047a49c9bcb410a9c458d0f: done: 0.030502 sec","job_status":"done","queue_duration_s":1.867166,"scheduling_latency_s":1.867185,"redis_calls":19,"redis_duration_s":0.002503,"redis_read_bytes":22,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000372,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005457399,"duration_s":0.030502,"completed_at":"2026-09-03T10:17:03.714Z","load_balancing_strategy":"primary","db_duration_s":0.006191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2382gitlab # [ 246.696158] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.715Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","06e6874a852a4c3815afd29f"],"class":"WebHooks::LogExecutionWorker","jid":"3aaf8254550ba8237e7e63c5","created_at":"2026-09-03T10:17:02.579Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-74e90fd054914d9a","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3060f59947a85a34429e989e0d248e7dc8d71827dfba0d17e841a21d633756f1","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:02.584Z","job_size_bytes":2593,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3aaf8254550ba8237e7e63c5: start","job_status":"start","queue_duration_s":1.130807,"scheduling_latency_s":1.130821}2383gitlab # [ 246.744901] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.744Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","06e6874a852a4c3815afd29f"],"class":"WebHooks::LogExecutionWorker","jid":"3aaf8254550ba8237e7e63c5","created_at":"2026-09-03T10:17:02.579Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-74e90fd054914d9a","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3060f59947a85a34429e989e0d248e7dc8d71827dfba0d17e841a21d633756f1","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:02.584Z","job_size_bytes":2593,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3aaf8254550ba8237e7e63c5: done: 0.029059 sec","job_status":"done","queue_duration_s":1.130807,"scheduling_latency_s":1.130821,"redis_calls":10,"redis_duration_s":0.00145,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000250032,"duration_s":0.029059,"completed_at":"2026-09-03T10:17:03.744Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2384gitlab # [ 246.750849] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.745Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d8d2993b38364f9222a22b86"],"class":"WebHooks::LogExecutionWorker","jid":"4f0ecbb5522480cf5ba7e28f","created_at":"2026-09-03T10:17:03.113Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-626e3bae979f4b5c","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0facfcddfe92fff39ae86ede72b7dc527b798447489596fcb42a97f696fa50f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.115Z","job_size_bytes":2592,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4f0ecbb5522480cf5ba7e28f: start","job_status":"start","queue_duration_s":0.629692,"scheduling_latency_s":0.629708}2385gitlab # [ 246.766219] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.763Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d8d2993b38364f9222a22b86"],"class":"WebHooks::LogExecutionWorker","jid":"4f0ecbb5522480cf5ba7e28f","created_at":"2026-09-03T10:17:03.113Z","trace_propagation_headers":{"sentry-trace":"8cbd9a0204b04ed5b528e4839ea7cc64-626e3bae979f4b5c","baggage":"sentry-trace_id=8cbd9a0204b04ed5b528e4839ea7cc64,sentry-release=Unknown"},"correlation_id":"01M1KC9QD2972HN4KTYCB33A5W","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0facfcddfe92fff39ae86ede72b7dc527b798447489596fcb42a97f696fa50f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.115Z","job_size_bytes":2592,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4f0ecbb5522480cf5ba7e28f: done: 0.018071 sec","job_status":"done","queue_duration_s":0.629692,"scheduling_latency_s":0.629708,"redis_calls":7,"redis_duration_s":0.002591,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002326,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018071,"completed_at":"2026-09-03T10:17:03.763Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002955,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2386gitlab # [ 246.811609] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.766Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"09f82184d8afb7cf0d92cc5b","created_at":"2026-09-03T10:17:03.525Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.526Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-09f82184d8afb7cf0d92cc5b: start","job_status":"start","queue_duration_s":0.239696,"scheduling_latency_s":0.239709}2387gitlab # [ 246.818395] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"09f82184d8afb7cf0d92cc5b","created_at":"2026-09-03T10:17:03.525Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.526Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-09f82184d8afb7cf0d92cc5b: done: 0.025771 sec","job_status":"done","queue_duration_s":0.239696,"scheduling_latency_s":0.239709,"redis_calls":7,"redis_duration_s":0.0010479999999999999,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025771,"completed_at":"2026-09-03T10:17:03.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005521,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2388gitlab # [ 246.867299] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.793Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"d1c5562b4bb3cb9e978318ab","created_at":"2026-09-03T10:17:03.568Z","trace_propagation_headers":{"sentry-trace":"b26de573a0114814abf45e879682d41b-ea55bdc4edac4df4","baggage":"sentry-trace_id=b26de573a0114814abf45e879682d41b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCB8P09GWG8TDVRMRHM6Z2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.570Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-d1c5562b4bb3cb9e978318ab: start","job_status":"start","queue_duration_s":0.222323,"scheduling_latency_s":0.222337}2389gitlab # [ 246.870726] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.809Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"d1c5562b4bb3cb9e978318ab","created_at":"2026-09-03T10:17:03.568Z","trace_propagation_headers":{"sentry-trace":"b26de573a0114814abf45e879682d41b-ea55bdc4edac4df4","baggage":"sentry-trace_id=b26de573a0114814abf45e879682d41b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCB8P09GWG8TDVRMRHM6Z2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.570Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-d1c5562b4bb3cb9e978318ab: done: 0.016743 sec","job_status":"done","queue_duration_s":0.222323,"scheduling_latency_s":0.222337,"redis_calls":7,"redis_duration_s":0.000896,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016743,"completed_at":"2026-09-03T10:17:03.809Z","load_balancing_strategy":"primary","db_duration_s":0.002922,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2390gitlab # [ 246.909160] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.810Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"72e07bb20efadb74a2841fd5","created_at":"2026-09-03T10:17:03.571Z","trace_propagation_headers":{"sentry-trace":"b26de573a0114814abf45e879682d41b-ea55bdc4edac4df4","baggage":"sentry-trace_id=b26de573a0114814abf45e879682d41b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCB8P09GWG8TDVRMRHM6Z2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.572Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-72e07bb20efadb74a2841fd5: start","job_status":"start","queue_duration_s":0.238135,"scheduling_latency_s":0.238149}2391gitlab # [ 246.924982] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.840Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"72e07bb20efadb74a2841fd5","created_at":"2026-09-03T10:17:03.571Z","trace_propagation_headers":{"sentry-trace":"b26de573a0114814abf45e879682d41b-ea55bdc4edac4df4","baggage":"sentry-trace_id=b26de573a0114814abf45e879682d41b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCB8P09GWG8TDVRMRHM6Z2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.572Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-72e07bb20efadb74a2841fd5: done: 0.029567 sec","job_status":"done","queue_duration_s":0.238135,"scheduling_latency_s":0.238149,"redis_calls":10,"redis_duration_s":0.001716,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001046,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029567,"completed_at":"2026-09-03T10:17:03.840Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2392gitlab # [ 246.968374] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.841Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4662d253489a65b853b72789","created_at":"2026-09-03T10:17:03.606Z","trace_propagation_headers":{"sentry-trace":"b26de573a0114814abf45e879682d41b-ea55bdc4edac4df4","baggage":"sentry-trace_id=b26de573a0114814abf45e879682d41b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCB8P09GWG8TDVRMRHM6Z2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.608Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4662d253489a65b853b72789: start","job_status":"start","queue_duration_s":0.232398,"scheduling_latency_s":0.232413}2393gitlab # [ 246.976083] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.916Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4662d253489a65b853b72789","created_at":"2026-09-03T10:17:03.606Z","trace_propagation_headers":{"sentry-trace":"b26de573a0114814abf45e879682d41b-ea55bdc4edac4df4","baggage":"sentry-trace_id=b26de573a0114814abf45e879682d41b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCB8P09GWG8TDVRMRHM6Z2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.608Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4662d253489a65b853b72789: done: 0.075872 sec","job_status":"done","queue_duration_s":0.232398,"scheduling_latency_s":0.232413,"gitaly_calls":2,"gitaly_duration_s":0.004623,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004033,"redis_read_bytes":80,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00054,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000774,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.00018,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000139,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000391,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054232,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05722934,"duration_s":0.075872,"completed_at":"2026-09-03T10:17:03.916Z","load_balancing_strategy":"primary","db_duration_s":0.017032,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2394gitlab # [ 247.047322] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.918Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c1364ace56f5f6e7e191a8c7","created_at":"2026-09-03T10:17:03.613Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.615Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c1364ace56f5f6e7e191a8c7: start","job_status":"start","queue_duration_s":0.3022,"scheduling_latency_s":0.302216}2395gitlab # [ 247.047873] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.933Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c1364ace56f5f6e7e191a8c7","created_at":"2026-09-03T10:17:03.613Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.615Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c1364ace56f5f6e7e191a8c7: done: 0.015822 sec","job_status":"done","queue_duration_s":0.3022,"scheduling_latency_s":0.302216,"redis_calls":7,"redis_duration_s":0.000867,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000238,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015822,"completed_at":"2026-09-03T10:17:03.933Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002387,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2396gitlab # [ 247.100899] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.934Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b5c1d2031271c22408549f29","created_at":"2026-09-03T10:17:03.620Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.622Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b5c1d2031271c22408549f29: start","job_status":"start","queue_duration_s":0.312555,"scheduling_latency_s":0.312568}2397gitlab # [ 247.101600] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.950Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b5c1d2031271c22408549f29","created_at":"2026-09-03T10:17:03.620Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.622Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b5c1d2031271c22408549f29: done: 0.015296 sec","job_status":"done","queue_duration_s":0.312555,"scheduling_latency_s":0.312568,"redis_calls":7,"redis_duration_s":0.000874,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015296,"completed_at":"2026-09-03T10:17:03.949Z","load_balancing_strategy":"primary","db_duration_s":0.002475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2398gitlab # [ 247.141444] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.950Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ffa85d1812e11a62cf465e90","created_at":"2026-09-03T10:17:03.627Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.629Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ffa85d1812e11a62cf465e90: start","job_status":"start","queue_duration_s":0.320931,"scheduling_latency_s":0.320945}2399gitlab # [ 247.142088] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.967Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ffa85d1812e11a62cf465e90","created_at":"2026-09-03T10:17:03.627Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.629Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ffa85d1812e11a62cf465e90: done: 0.016827 sec","job_status":"done","queue_duration_s":0.320931,"scheduling_latency_s":0.320945,"redis_calls":10,"redis_duration_s":0.001218,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000628,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000351,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016827,"completed_at":"2026-09-03T10:17:03.967Z","load_balancing_strategy":"primary","db_duration_s":0.002538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2400gitlab # [ 247.189734] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.968Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5c31b19870f599f2aba0409b","created_at":"2026-09-03T10:17:03.630Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.631Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5c31b19870f599f2aba0409b: start","job_status":"start","queue_duration_s":0.337145,"scheduling_latency_s":0.337159}2401gitlab # [ 247.190533] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.983Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5c31b19870f599f2aba0409b","created_at":"2026-09-03T10:17:03.630Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.631Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5c31b19870f599f2aba0409b: done: 0.014607 sec","job_status":"done","queue_duration_s":0.337145,"scheduling_latency_s":0.337159,"redis_calls":7,"redis_duration_s":0.00092,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014607,"completed_at":"2026-09-03T10:17:03.983Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2402gitlab # [ 247.237751] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:03.984Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"76920e9d709b2ba1d47995e8","created_at":"2026-09-03T10:17:03.631Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.633Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-76920e9d709b2ba1d47995e8: start","job_status":"start","queue_duration_s":0.350961,"scheduling_latency_s":0.350975}2403gitlab # [ 247.239699] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.002Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"76920e9d709b2ba1d47995e8","created_at":"2026-09-03T10:17:03.631Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.633Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-76920e9d709b2ba1d47995e8: done: 0.018367 sec","job_status":"done","queue_duration_s":0.350961,"scheduling_latency_s":0.350975,"redis_calls":8,"redis_duration_s":0.001376,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000301,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018367,"completed_at":"2026-09-03T10:17:04.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002607,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2404gitlab # [ 247.303421] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.003Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a36ef59ac9e33a4e34eb870d","created_at":"2026-09-03T10:17:03.633Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.634Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a36ef59ac9e33a4e34eb870d: start","job_status":"start","queue_duration_s":0.368746,"scheduling_latency_s":0.368776}2405gitlab # [ 247.304058] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.017Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a36ef59ac9e33a4e34eb870d","created_at":"2026-09-03T10:17:03.633Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.634Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a36ef59ac9e33a4e34eb870d: done: 0.014132 sec","job_status":"done","queue_duration_s":0.368746,"scheduling_latency_s":0.368776,"redis_calls":7,"redis_duration_s":0.000899,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014132,"completed_at":"2026-09-03T10:17:04.017Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001638,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2406gitlab # [ 247.331739] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.018Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"343491f716c545ae0587b586","created_at":"2026-09-03T10:17:03.634Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.635Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-343491f716c545ae0587b586: start","job_status":"start","queue_duration_s":0.382517,"scheduling_latency_s":0.38253}2407gitlab # [ 247.374645] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.033Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"343491f716c545ae0587b586","created_at":"2026-09-03T10:17:03.634Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.635Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-343491f716c545ae0587b586: done: 0.015358 sec","job_status":"done","queue_duration_s":0.382517,"scheduling_latency_s":0.38253,"redis_calls":7,"redis_duration_s":0.000923,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015358,"completed_at":"2026-09-03T10:17:04.033Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2408gitlab # [ 247.416192] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.034Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"735779b32a687aae774b5316","created_at":"2026-09-03T10:17:03.636Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.637Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-735779b32a687aae774b5316: start","job_status":"start","queue_duration_s":0.397584,"scheduling_latency_s":0.397598}2409gitlab # [ 247.416901] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.042Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"735779b32a687aae774b5316","created_at":"2026-09-03T10:17:03.636Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.637Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-735779b32a687aae774b5316: done: 0.008171 sec","job_status":"done","queue_duration_s":0.397584,"scheduling_latency_s":0.397598,"redis_calls":7,"redis_duration_s":0.0008990000000000001,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008171,"completed_at":"2026-09-03T10:17:04.042Z","load_balancing_strategy":"primary","db_duration_s":0.000244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2410gitlab # [ 247.456335] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.043Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"ea1fb051fce9130fb339a825","created_at":"2026-09-03T10:17:03.637Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.638Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-ea1fb051fce9130fb339a825: start","job_status":"start","queue_duration_s":0.405446,"scheduling_latency_s":0.405459}2411gitlab # [ 247.456939] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.056Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"ea1fb051fce9130fb339a825","created_at":"2026-09-03T10:17:03.637Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.638Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-ea1fb051fce9130fb339a825: done: 0.012911 sec","job_status":"done","queue_duration_s":0.405446,"scheduling_latency_s":0.405459,"redis_calls":7,"redis_duration_s":0.0009109999999999999,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012911,"completed_at":"2026-09-03T10:17:04.056Z","load_balancing_strategy":"primary","db_duration_s":0.001846,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2412gitlab # [ 247.508081] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.057Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"c892308334f08cf2b0ec3522","created_at":"2026-09-03T10:17:03.678Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.679Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c892308334f08cf2b0ec3522: start","job_status":"start","queue_duration_s":0.377889,"scheduling_latency_s":0.377903}2413gitlab # [ 247.508334] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.071Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"c892308334f08cf2b0ec3522","created_at":"2026-09-03T10:17:03.678Z","trace_propagation_headers":{"sentry-trace":"23dfd03185bb4152b186c637a1e70d2a-5c3e142a73044adf","baggage":"sentry-trace_id=23dfd03185bb4152b186c637a1e70d2a,sentry-release=Unknown"},"correlation_id":"01M1KC9RA7SASYBQ7BHQKAM95X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.679Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c892308334f08cf2b0ec3522: done: 0.014211 sec","job_status":"done","queue_duration_s":0.377889,"scheduling_latency_s":0.377903,"redis_calls":8,"redis_duration_s":0.001027,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000229,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014211,"completed_at":"2026-09-03T10:17:04.071Z","load_balancing_strategy":"primary","db_duration_s":0.001947,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2414gitlab # [ 247.555840] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.072Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f9716850ac04592e931fa6db","created_at":"2026-09-03T10:17:03.705Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-34a61c39d5af497f","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.709Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f9716850ac04592e931fa6db: start","job_status":"start","queue_duration_s":0.363047,"scheduling_latency_s":0.363061}2415gitlab # [ 247.556732] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.123Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f9716850ac04592e931fa6db","created_at":"2026-09-03T10:17:03.705Z","trace_propagation_headers":{"sentry-trace":"6f9e5c7834ba461494cdd7afe0a7ddd7-34a61c39d5af497f","baggage":"sentry-trace_id=6f9e5c7834ba461494cdd7afe0a7ddd7,sentry-release=Unknown"},"correlation_id":"01M1KC9M8RN7T7TW2X3MGAKXQ9","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.709Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f9716850ac04592e931fa6db: done: 0.050677 sec","job_status":"done","queue_duration_s":0.363047,"scheduling_latency_s":0.363061,"redis_calls":11,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208127,"duration_s":0.050677,"completed_at":"2026-09-03T10:17:04.123Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01213,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2416gitlab # [ 247.606605] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.124Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"41f221cac88eea637a5c800c","created_at":"2026-09-03T10:17:03.874Z","trace_propagation_headers":{"sentry-trace":"b26de573a0114814abf45e879682d41b-b7bedd1261a64eab","baggage":"sentry-trace_id=b26de573a0114814abf45e879682d41b,sentry-release=Unknown"},"correlation_id":"01M1KCB8P09GWG8TDVRMRHM6Z2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.875Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-41f221cac88eea637a5c800c: start","job_status":"start","queue_duration_s":0.248706,"scheduling_latency_s":0.24872}2417gitlab # [ 247.607259] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:04.143Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"41f221cac88eea637a5c800c","created_at":"2026-09-03T10:17:03.874Z","trace_propagation_headers":{"sentry-trace":"b26de573a0114814abf45e879682d41b-b7bedd1261a64eab","baggage":"sentry-trace_id=b26de573a0114814abf45e879682d41b,sentry-release=Unknown"},"correlation_id":"01M1KCB8P09GWG8TDVRMRHM6Z2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:03.875Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-41f221cac88eea637a5c800c: done: 0.019153 sec","job_status":"done","queue_duration_s":0.248706,"scheduling_latency_s":0.24872,"redis_calls":7,"redis_duration_s":0.0008900000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019153,"completed_at":"2026-09-03T10:17:04.143Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2418gitlab # [ 247.828236] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 8223996324 wd_nsec: 82239955932419gitlab # [ 247.698127] nixbot[2597]: INFO: ::ffff:127.0.0.1:59976 - "GET /api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 200 OK2420gitlab # [ 247.740702] nix-daemon[2825]: accepted connection from pid 3380, user nixbot2421gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=eba459b7c8f87c78f7a7240ca13a81922b9f7662', in 3.45 seconds)2422(finished: subtest: push delivers the webhook, in 54.99 seconds)2423subtest: eval and build post commit statuses2424gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses2425gitlab # [ 247.856433] gitaly[920]: time="2026-09-03T10:17:07.844Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBCXX793A025AY9XJZA2R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:37.743" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:07.843" grpc.time_ms=0.74 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2426gitlab # [ 247.894285] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1082427gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses, in 0.13 seconds)2428gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses2429gitlab # [ 248.994639] gitaly[920]: time="2026-09-03T10:17:08.982Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBE1V3YHNCSGF8EPN6EZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:38.881" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:08.981" grpc.time_ms=0.816 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2430gitlab # [ 249.049570] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1142431gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses, in 0.15 seconds)2432gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses2433gitlab # [ 250.148639] gitaly[920]: time="2026-09-03T10:17:10.136Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBF5YPWH80AG84YZRZ1P5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:40.035" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:10.135" grpc.time_ms=0.857 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2434gitlab # [ 250.176580] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 862435gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses, in 0.13 seconds)2436gitlab # [ 250.593299] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:10.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8901e7bc16f877a8a5db87ac","created_at":"2026-09-03T10:17:10.565Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.578Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8901e7bc16f877a8a5db87ac: start","job_status":"start","queue_duration_s":0.002585,"scheduling_latency_s":0.002633}2437gitlab # [ 250.948237] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:10.936Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8901e7bc16f877a8a5db87ac","created_at":"2026-09-03T10:17:10.565Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.578Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8901e7bc16f877a8a5db87ac: done: 0.35541 sec","job_status":"done","queue_duration_s":0.002585,"scheduling_latency_s":0.002633,"redis_calls":62,"redis_duration_s":0.13081,"redis_read_bytes":110,"redis_write_bytes":41440,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002035,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_calls":2,"redis_queues_duration_s":0.001213,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.123305,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004257,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.172245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.35541,"completed_at":"2026-09-03T10:17:10.936Z","load_balancing_strategy":"primary","db_duration_s":0.004229,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2438gitlab # [ 251.016771] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.005Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5ac73f1686d34bda7e980a45","created_at":"2026-09-03T10:17:10.587Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b27fafa964f327ca23162fe07b2bed8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.601Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5ac73f1686d34bda7e980a45: start","job_status":"start","queue_duration_s":0.404111,"scheduling_latency_s":0.404158}2439gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses2440gitlab # [ 251.214402] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5ac73f1686d34bda7e980a45","created_at":"2026-09-03T10:17:10.587Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b27fafa964f327ca23162fe07b2bed8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.601Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5ac73f1686d34bda7e980a45: done: 0.197523 sec","job_status":"done","queue_duration_s":0.404111,"scheduling_latency_s":0.404158,"redis_calls":13,"redis_duration_s":0.020644,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003836,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012019,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004789,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.038,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002594463,"exclusive_lock_hold_duration_s":0.079012557,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.197523,"completed_at":"2026-09-03T10:17:11.202Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056552,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2441gitlab # [ 251.279212] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a5b0e1ebe9034117c3a9ae24","created_at":"2026-09-03T10:17:10.615Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f67b0492a7dccd64f0abb123f4f4feb","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.636Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a5b0e1ebe9034117c3a9ae24: start","job_status":"start","queue_duration_s":0.630009,"scheduling_latency_s":0.630053}2442gitlab # [ 251.304664] gitaly[920]: time="2026-09-03T10:17:11.293Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBG9GJNFZFSRSPACZBN99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:41.192" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:11.292" grpc.time_ms=0.502 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2443gitlab # [ 251.332393] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 582444gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses, in 0.15 seconds)2445gitlab # [ 251.363595] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.352Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a5b0e1ebe9034117c3a9ae24","created_at":"2026-09-03T10:17:10.615Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f67b0492a7dccd64f0abb123f4f4feb","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.636Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a5b0e1ebe9034117c3a9ae24: done: 0.085518 sec","job_status":"done","queue_duration_s":0.630009,"scheduling_latency_s":0.630053,"redis_calls":12,"redis_duration_s":0.014719999999999999,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01151,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001805,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.012683456,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.085518,"completed_at":"2026-09-03T10:17:11.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007115,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2446gitlab # [ 251.406296] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e1c14a157d14196ef9aa4279","created_at":"2026-09-03T10:17:10.649Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e71da517ab09b2289039a1cf39a58e5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.662Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e1c14a157d14196ef9aa4279: start","job_status":"start","queue_duration_s":0.732118,"scheduling_latency_s":0.732162}2447gitlab # [ 251.447634] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.436Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2448gitlab # [ 251.479227] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e1c14a157d14196ef9aa4279","created_at":"2026-09-03T10:17:10.649Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e71da517ab09b2289039a1cf39a58e5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.662Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e1c14a157d14196ef9aa4279: done: 0.072492 sec","job_status":"done","queue_duration_s":0.732118,"scheduling_latency_s":0.732162,"redis_calls":9,"redis_duration_s":0.00993,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001894,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003707,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.072492,"completed_at":"2026-09-03T10:17:11.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2449gitlab # [ 251.533556] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.522Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a627c907c1605dabb833c545","created_at":"2026-09-03T10:17:10.681Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"291a59f31f201785e053879c653cb4f5","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.693Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a627c907c1605dabb833c545: start","job_status":"start","queue_duration_s":0.828585,"scheduling_latency_s":0.82863}2450gitlab # [ 251.598698] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.586Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a627c907c1605dabb833c545","created_at":"2026-09-03T10:17:10.681Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"291a59f31f201785e053879c653cb4f5","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.693Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a627c907c1605dabb833c545: done: 0.064446 sec","job_status":"done","queue_duration_s":0.828585,"scheduling_latency_s":0.82863,"redis_calls":9,"redis_duration_s":0.008374,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002379,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.064446,"completed_at":"2026-09-03T10:17:11.586Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2451gitlab # [ 251.654507] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"37e7659bc4de0a41db89476c","created_at":"2026-09-03T10:17:10.708Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f8d9e0620ad5b797c2f44c6aa966a20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:17:10.719Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-37e7659bc4de0a41db89476c: start","job_status":"start","queue_duration_s":0.923378,"scheduling_latency_s":0.923425}2452gitlab # [ 251.672466] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.661Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2453gitlab # [ 251.682411] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.670Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"37e7659bc4de0a41db89476c","created_at":"2026-09-03T10:17:10.708Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f8d9e0620ad5b797c2f44c6aa966a20","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:17:10.719Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-37e7659bc4de0a41db89476c: done: 0.028001 sec","job_status":"done","queue_duration_s":0.923378,"scheduling_latency_s":0.923425,"redis_calls":9,"redis_duration_s":0.002935,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000653,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001658,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000624,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028001,"completed_at":"2026-09-03T10:17:11.670Z","load_balancing_strategy":"primary","db_duration_s":0.000824,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2454gitlab # [ 251.684443] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"37ce0e5ddbf089f471d12140","created_at":"2026-09-03T10:17:10.792Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"949b520fb56f75d07af97ff2295c0e07","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.803Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-37ce0e5ddbf089f471d12140: start","job_status":"start","queue_duration_s":0.869486,"scheduling_latency_s":0.869517}2455gitlab # [ 251.698976] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.687Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2456gitlab # [ 251.706887] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"37ce0e5ddbf089f471d12140","created_at":"2026-09-03T10:17:10.792Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"949b520fb56f75d07af97ff2295c0e07","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.803Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-37ce0e5ddbf089f471d12140: done: 0.022419 sec","job_status":"done","queue_duration_s":0.869486,"scheduling_latency_s":0.869517,"redis_calls":9,"redis_duration_s":0.002593,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000603,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001364,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000626,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022419,"completed_at":"2026-09-03T10:17:11.695Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000608,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2457gitlab # [ 251.735737] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3c08eac2418bf64fc25ee444","created_at":"2026-09-03T10:17:10.847Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1eb27a6ffb9f3c09a66235e46327462","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.855Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3c08eac2418bf64fc25ee444: start","job_status":"start","queue_duration_s":0.841232,"scheduling_latency_s":0.841258}2458gitlab # [ 251.784142] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3c08eac2418bf64fc25ee444","created_at":"2026-09-03T10:17:10.847Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1eb27a6ffb9f3c09a66235e46327462","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.855Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3c08eac2418bf64fc25ee444: done: 0.030682 sec","job_status":"done","queue_duration_s":0.841232,"scheduling_latency_s":0.841258,"redis_calls":11,"redis_duration_s":0.002641,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000473244,"exclusive_lock_hold_duration_s":0.01192442,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.030682,"completed_at":"2026-09-03T10:17:11.727Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2459gitlab # [ 251.788136] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dbe183eacde0bc86baf0c609","created_at":"2026-09-03T10:17:10.870Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57086c7b27273edf3a8b8adc765e0982","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.880Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dbe183eacde0bc86baf0c609: start","job_status":"start","queue_duration_s":0.8489,"scheduling_latency_s":0.848913}2460gitlab # [ 251.800625] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.744Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dbe183eacde0bc86baf0c609","created_at":"2026-09-03T10:17:10.870Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57086c7b27273edf3a8b8adc765e0982","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.880Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dbe183eacde0bc86baf0c609: done: 0.016069 sec","job_status":"done","queue_duration_s":0.8489,"scheduling_latency_s":0.848913,"redis_calls":9,"redis_duration_s":0.0024089999999999997,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001194,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000725,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016069,"completed_at":"2026-09-03T10:17:11.744Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2461gitlab # [ 251.843121] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"5a4b29efc3c41e969c7ec619","created_at":"2026-09-03T10:17:10.889Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aeaf4663195e53bf69320a382f4f6ef8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.898Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-5a4b29efc3c41e969c7ec619: start","job_status":"start","queue_duration_s":0.847372,"scheduling_latency_s":0.847386}2462gitlab # [ 251.885170] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.770Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"5a4b29efc3c41e969c7ec619","created_at":"2026-09-03T10:17:10.889Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aeaf4663195e53bf69320a382f4f6ef8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.898Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-5a4b29efc3c41e969c7ec619: done: 0.024722 sec","job_status":"done","queue_duration_s":0.847372,"scheduling_latency_s":0.847386,"redis_calls":11,"redis_duration_s":0.002613,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000875,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000607,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024722,"completed_at":"2026-09-03T10:17:11.770Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2463gitlab # [ 251.887876] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"008b65fed5fd6d0138995d9b","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-008b65fed5fd6d0138995d9b: start","job_status":"start","queue_duration_s":0.85508,"scheduling_latency_s":0.85511}2464gitlab # [ 251.899838] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"008b65fed5fd6d0138995d9b","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-008b65fed5fd6d0138995d9b: done: 0.02533 sec","job_status":"done","queue_duration_s":0.85508,"scheduling_latency_s":0.85511,"redis_calls":13,"redis_duration_s":0.003944,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002699,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02533,"completed_at":"2026-09-03T10:17:11.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2465gitlab # [ 251.941309] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c5cc354fa97a05874da39e7","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c5cc354fa97a05874da39e7: start","job_status":"start","queue_duration_s":0.882626,"scheduling_latency_s":0.882664}2466gitlab # [ 251.944822] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c5cc354fa97a05874da39e7","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c5cc354fa97a05874da39e7: done: 0.020455 sec","job_status":"done","queue_duration_s":0.882626,"scheduling_latency_s":0.882664,"redis_calls":13,"redis_duration_s":0.0035199999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002365,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020455,"completed_at":"2026-09-03T10:17:11.820Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2467gitlab # [ 251.986076] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80ee2bf1688a4e8a2e60a99d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80ee2bf1688a4e8a2e60a99d: start","job_status":"start","queue_duration_s":0.904677,"scheduling_latency_s":0.904702}2468gitlab # [ 251.990689] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80ee2bf1688a4e8a2e60a99d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80ee2bf1688a4e8a2e60a99d: done: 0.013121 sec","job_status":"done","queue_duration_s":0.904677,"scheduling_latency_s":0.904702,"redis_calls":13,"redis_duration_s":0.002694,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013121,"completed_at":"2026-09-03T10:17:11.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2469gitlab # [ 252.037436] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49879fcfa94fbc09cdacc9bf","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49879fcfa94fbc09cdacc9bf: start","job_status":"start","queue_duration_s":0.91961,"scheduling_latency_s":0.919638}2470gitlab # [ 252.038366] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49879fcfa94fbc09cdacc9bf","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49879fcfa94fbc09cdacc9bf: done: 0.013096 sec","job_status":"done","queue_duration_s":0.91961,"scheduling_latency_s":0.919638,"redis_calls":13,"redis_duration_s":0.002842,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001762,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013096,"completed_at":"2026-09-03T10:17:11.850Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2471gitlab # [ 252.072683] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e43b37cd1ab193938ab129","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e43b37cd1ab193938ab129: start","job_status":"start","queue_duration_s":0.933729,"scheduling_latency_s":0.933742}2472gitlab # [ 252.083672] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e43b37cd1ab193938ab129","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e43b37cd1ab193938ab129: done: 0.010336 sec","job_status":"done","queue_duration_s":0.933729,"scheduling_latency_s":0.933742,"redis_calls":13,"redis_duration_s":0.002656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001755,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010336,"completed_at":"2026-09-03T10:17:11.861Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2473gitlab # [ 252.128687] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"669ffae9c6fd1b27467ee6ed","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-669ffae9c6fd1b27467ee6ed: start","job_status":"start","queue_duration_s":0.945122,"scheduling_latency_s":0.945135}2474gitlab # [ 252.129681] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"669ffae9c6fd1b27467ee6ed","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-669ffae9c6fd1b27467ee6ed: done: 0.010228 sec","job_status":"done","queue_duration_s":0.945122,"scheduling_latency_s":0.945135,"redis_calls":13,"redis_duration_s":0.0026060000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001806,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010228,"completed_at":"2026-09-03T10:17:11.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2475gitlab # [ 252.164862] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"350299b98a66beb998b0cda7","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-350299b98a66beb998b0cda7: start","job_status":"start","queue_duration_s":0.956569,"scheduling_latency_s":0.956583}2476gitlab # [ 252.204978] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"350299b98a66beb998b0cda7","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-350299b98a66beb998b0cda7: done: 0.010108 sec","job_status":"done","queue_duration_s":0.956569,"scheduling_latency_s":0.956583,"redis_calls":13,"redis_duration_s":0.002589,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001789,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010108,"completed_at":"2026-09-03T10:17:11.884Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2477gitlab # [ 252.209877] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3a01b8c7100f77c948953f2","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3a01b8c7100f77c948953f2: start","job_status":"start","queue_duration_s":0.967761,"scheduling_latency_s":0.967774}2478gitlab # [ 252.221779] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3a01b8c7100f77c948953f2","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3a01b8c7100f77c948953f2: done: 0.020937 sec","job_status":"done","queue_duration_s":0.967761,"scheduling_latency_s":0.967774,"redis_calls":13,"redis_duration_s":0.003115,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020937,"completed_at":"2026-09-03T10:17:11.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2479gitlab # [ 252.268502] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21e45b7ee179b6a957f0a90d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21e45b7ee179b6a957f0a90d: start","job_status":"start","queue_duration_s":0.990799,"scheduling_latency_s":0.99083}2480gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses2481gitlab # [ 252.269818] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21e45b7ee179b6a957f0a90d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21e45b7ee179b6a957f0a90d: done: 0.017439 sec","job_status":"done","queue_duration_s":0.990799,"scheduling_latency_s":0.99083,"redis_calls":13,"redis_duration_s":0.002275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001516,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017439,"completed_at":"2026-09-03T10:17:11.925Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2482gitlab # [ 252.340981] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f29ad57e09752a4ca77bdb64","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f29ad57e09752a4ca77bdb64: start","job_status":"start","queue_duration_s":1.00908,"scheduling_latency_s":1.009093}2483gitlab # [ 252.341663] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f29ad57e09752a4ca77bdb64","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f29ad57e09752a4ca77bdb64: done: 0.020531 sec","job_status":"done","queue_duration_s":1.00908,"scheduling_latency_s":1.009093,"redis_calls":13,"redis_duration_s":0.003004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001744,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020531,"completed_at":"2026-09-03T10:17:11.947Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2484gitlab # [ 252.347610] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9334f64f25b9f04d75ad446d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9334f64f25b9f04d75ad446d: start","job_status":"start","queue_duration_s":1.031803,"scheduling_latency_s":1.03184}2485gitlab # [ 252.357887] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9334f64f25b9f04d75ad446d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9334f64f25b9f04d75ad446d: done: 0.019624 sec","job_status":"done","queue_duration_s":1.031803,"scheduling_latency_s":1.03184,"redis_calls":13,"redis_duration_s":0.003269,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002291,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019624,"completed_at":"2026-09-03T10:17:11.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2486gitlab # [ 252.403942] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"289f7cea242a8d728649a8c7","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-289f7cea242a8d728649a8c7: start","job_status":"start","queue_duration_s":1.052567,"scheduling_latency_s":1.052585}2487gitlab # [ 252.404678] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"289f7cea242a8d728649a8c7","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-289f7cea242a8d728649a8c7: done: 0.010375 sec","job_status":"done","queue_duration_s":1.052567,"scheduling_latency_s":1.052585,"redis_calls":13,"redis_duration_s":0.002412,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001514,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010375,"completed_at":"2026-09-03T10:17:11.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2488gitlab # [ 252.445983] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b3d398e06ab298090bdbf6a","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b3d398e06ab298090bdbf6a: start","job_status":"start","queue_duration_s":1.064835,"scheduling_latency_s":1.064877}2489gitlab # [ 252.450762] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b3d398e06ab298090bdbf6a","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b3d398e06ab298090bdbf6a: done: 0.011327 sec","job_status":"done","queue_duration_s":1.064835,"scheduling_latency_s":1.064877,"redis_calls":13,"redis_duration_s":0.0022040000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001515,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011327,"completed_at":"2026-09-03T10:17:11.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2490gitlab # [ 252.483511] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:11.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6482261977b954f9325340df","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6482261977b954f9325340df: start","job_status":"start","queue_duration_s":1.077124,"scheduling_latency_s":1.077137}2491gitlab # [ 252.530495] nix-daemon[2825]: accepted connection from pid 3417, user nixbot2492gitlab # [ 252.531120] nix-daemon[2825]: accepted connection from pid 3380, user nixbot2493gitlab # [ 252.531630] gitaly[920]: time="2026-09-03T10:17:12.483Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHF9VPC0D2SGFHAYTXKG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:42.382" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:12.483" grpc.time_ms=0.367 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2494gitlab # [ 252.532709] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6482261977b954f9325340df","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6482261977b954f9325340df: done: 0.009756 sec","job_status":"done","queue_duration_s":1.077124,"scheduling_latency_s":1.077137,"redis_calls":13,"redis_duration_s":0.002154,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000762,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001392,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009756,"completed_at":"2026-09-03T10:17:12.004Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2495gitlab # [ 252.541653] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9d029a31753f78c78641448","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9d029a31753f78c78641448: start","job_status":"start","queue_duration_s":1.088031,"scheduling_latency_s":1.088044}2496gitlab # [ 252.552925] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9d029a31753f78c78641448","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9d029a31753f78c78641448: done: 0.01644 sec","job_status":"done","queue_duration_s":1.088031,"scheduling_latency_s":1.088044,"redis_calls":13,"redis_duration_s":0.002459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001625,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01644,"completed_at":"2026-09-03T10:17:12.022Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2497gitlab # [ 252.593801] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a3f0958c358563137e19e8f","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a3f0958c358563137e19e8f: start","job_status":"start","queue_duration_s":1.106037,"scheduling_latency_s":1.106065}2498gitlab # [ 252.597682] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a3f0958c358563137e19e8f","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a3f0958c358563137e19e8f: done: 0.012092 sec","job_status":"done","queue_duration_s":1.106037,"scheduling_latency_s":1.106065,"redis_calls":13,"redis_duration_s":0.001625,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012092,"completed_at":"2026-09-03T10:17:12.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2499gitlab # [ 252.636958] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03855706e5f528277cdb3472","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03855706e5f528277cdb3472: start","job_status":"start","queue_duration_s":1.118949,"scheduling_latency_s":1.118961}2500gitlab # [ 252.642806] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03855706e5f528277cdb3472","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03855706e5f528277cdb3472: done: 0.009245 sec","job_status":"done","queue_duration_s":1.118949,"scheduling_latency_s":1.118961,"redis_calls":13,"redis_duration_s":0.0016150000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009245,"completed_at":"2026-09-03T10:17:12.045Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2501gitlab # [ 252.695340] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd3bab90e8e7e5ac1f3a2777","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd3bab90e8e7e5ac1f3a2777: start","job_status":"start","queue_duration_s":1.129002,"scheduling_latency_s":1.129016}2502gitlab # [ 252.696059] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd3bab90e8e7e5ac1f3a2777","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd3bab90e8e7e5ac1f3a2777: done: 0.00952 sec","job_status":"done","queue_duration_s":1.129002,"scheduling_latency_s":1.129016,"redis_calls":13,"redis_duration_s":0.001563,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00952,"completed_at":"2026-09-03T10:17:12.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2503gitlab # [ 252.740628] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b349b6ef2ac201a3f2a711d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b349b6ef2ac201a3f2a711d: start","job_status":"start","queue_duration_s":1.139374,"scheduling_latency_s":1.139387}2504gitlab # [ 252.741245] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b349b6ef2ac201a3f2a711d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b349b6ef2ac201a3f2a711d: done: 0.009475 sec","job_status":"done","queue_duration_s":1.139374,"scheduling_latency_s":1.139387,"redis_calls":13,"redis_duration_s":0.0016920000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009475,"completed_at":"2026-09-03T10:17:12.066Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2505gitlab # [ 252.780039] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74459ca9c18d43cce9638fbb","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74459ca9c18d43cce9638fbb: start","job_status":"start","queue_duration_s":1.149799,"scheduling_latency_s":1.149811}2506gitlab # [ 252.780666] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74459ca9c18d43cce9638fbb","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74459ca9c18d43cce9638fbb: done: 0.009636 sec","job_status":"done","queue_duration_s":1.149799,"scheduling_latency_s":1.149811,"redis_calls":13,"redis_duration_s":0.00203,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001379,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009636,"completed_at":"2026-09-03T10:17:12.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2507gitlab # [ 252.821796] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5990067a1b0ea4041a93945f","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5990067a1b0ea4041a93945f: start","job_status":"start","queue_duration_s":1.160425,"scheduling_latency_s":1.160437}2508gitlab # [ 252.822684] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5990067a1b0ea4041a93945f","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5990067a1b0ea4041a93945f: done: 0.009762 sec","job_status":"done","queue_duration_s":1.160425,"scheduling_latency_s":1.160437,"redis_calls":13,"redis_duration_s":0.002186,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009762,"completed_at":"2026-09-03T10:17:12.087Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2509gitlab # [ 252.865783] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5fa6b39821ec1422bcbb380","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5fa6b39821ec1422bcbb380: start","job_status":"start","queue_duration_s":1.171313,"scheduling_latency_s":1.171325}2510gitlab # [ 252.866531] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5fa6b39821ec1422bcbb380","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5fa6b39821ec1422bcbb380: done: 0.009856 sec","job_status":"done","queue_duration_s":1.171313,"scheduling_latency_s":1.171325,"redis_calls":13,"redis_duration_s":0.002187,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001485,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009856,"completed_at":"2026-09-03T10:17:12.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2511gitlab # [ 252.926538] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e16ebe78913c63136ba6e412","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e16ebe78913c63136ba6e412: start","job_status":"start","queue_duration_s":1.182159,"scheduling_latency_s":1.182173}2512gitlab # [ 252.927255] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e16ebe78913c63136ba6e412","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e16ebe78913c63136ba6e412: done: 0.017015 sec","job_status":"done","queue_duration_s":1.182159,"scheduling_latency_s":1.182173,"redis_calls":13,"redis_duration_s":0.00237,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001644,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017015,"completed_at":"2026-09-03T10:17:12.116Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2513gitlab # [ 252.946499] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac519a20e1d0dd1bb3aa47d1","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac519a20e1d0dd1bb3aa47d1: start","job_status":"start","queue_duration_s":1.200307,"scheduling_latency_s":1.200321}2514gitlab # [ 252.957787] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac519a20e1d0dd1bb3aa47d1","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac519a20e1d0dd1bb3aa47d1: done: 0.010187 sec","job_status":"done","queue_duration_s":1.200307,"scheduling_latency_s":1.200321,"redis_calls":13,"redis_duration_s":0.0025239999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001736,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010187,"completed_at":"2026-09-03T10:17:12.128Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2515gitlab # [ 252.993775] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca34c5ac924dbdf31bcad322","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca34c5ac924dbdf31bcad322: start","job_status":"start","queue_duration_s":1.211547,"scheduling_latency_s":1.21156}2516gitlab # [ 253.005268] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca34c5ac924dbdf31bcad322","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca34c5ac924dbdf31bcad322: done: 0.009895 sec","job_status":"done","queue_duration_s":1.211547,"scheduling_latency_s":1.21156,"redis_calls":13,"redis_duration_s":0.002435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001666,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009895,"completed_at":"2026-09-03T10:17:12.138Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2517gitlab # [ 253.044101] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba6bb61c5db8ca4302ca91bd","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba6bb61c5db8ca4302ca91bd: start","job_status":"start","queue_duration_s":1.222462,"scheduling_latency_s":1.222474}2518gitlab # [ 253.050705] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba6bb61c5db8ca4302ca91bd","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba6bb61c5db8ca4302ca91bd: done: 0.010649 sec","job_status":"done","queue_duration_s":1.222462,"scheduling_latency_s":1.222474,"redis_calls":13,"redis_duration_s":0.002969,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001125,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001844,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010649,"completed_at":"2026-09-03T10:17:12.150Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2519gitlab # [ 253.092791] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff01ab139995ba1ec36c32b6","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff01ab139995ba1ec36c32b6: start","job_status":"start","queue_duration_s":1.234164,"scheduling_latency_s":1.234177}2520gitlab # [ 253.096074] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff01ab139995ba1ec36c32b6","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff01ab139995ba1ec36c32b6: done: 0.00991 sec","job_status":"done","queue_duration_s":1.234164,"scheduling_latency_s":1.234177,"redis_calls":13,"redis_duration_s":0.00239,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001644,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00991,"completed_at":"2026-09-03T10:17:12.161Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2521gitlab # [ 253.161961] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"522af15575ea81462cbb6409","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-522af15575ea81462cbb6409: start","job_status":"start","queue_duration_s":1.245143,"scheduling_latency_s":1.245156}2522gitlab # [ 253.162726] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"522af15575ea81462cbb6409","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-522af15575ea81462cbb6409: done: 0.010035 sec","job_status":"done","queue_duration_s":1.245143,"scheduling_latency_s":1.245156,"redis_calls":13,"redis_duration_s":0.002545,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001721,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010035,"completed_at":"2026-09-03T10:17:12.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2523gitlab # [ 253.204520] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cacd587958f37eafb12e43e","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cacd587958f37eafb12e43e: start","job_status":"start","queue_duration_s":1.256234,"scheduling_latency_s":1.256247}2524gitlab # [ 253.205214] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cacd587958f37eafb12e43e","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cacd587958f37eafb12e43e: done: 0.010096 sec","job_status":"done","queue_duration_s":1.256234,"scheduling_latency_s":1.256247,"redis_calls":13,"redis_duration_s":0.002537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001733,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010096,"completed_at":"2026-09-03T10:17:12.183Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2525gitlab # [ 253.245764] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"913c6b2a479ba7a9a0d48477","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-913c6b2a479ba7a9a0d48477: start","job_status":"start","queue_duration_s":1.26737,"scheduling_latency_s":1.267384}2526gitlab # [ 253.246681] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"913c6b2a479ba7a9a0d48477","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-913c6b2a479ba7a9a0d48477: done: 0.010032 sec","job_status":"done","queue_duration_s":1.26737,"scheduling_latency_s":1.267384,"redis_calls":13,"redis_duration_s":0.002479,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010032,"completed_at":"2026-09-03T10:17:12.194Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2527gitlab # [ 253.298658] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d759d1986fe059be84474848","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d759d1986fe059be84474848: start","job_status":"start","queue_duration_s":1.27848,"scheduling_latency_s":1.278494}2528gitlab # [ 253.299571] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d759d1986fe059be84474848","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d759d1986fe059be84474848: done: 0.010176 sec","job_status":"done","queue_duration_s":1.27848,"scheduling_latency_s":1.278494,"redis_calls":13,"redis_duration_s":0.002571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010176,"completed_at":"2026-09-03T10:17:12.206Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2529gitlab # [ 253.337666] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b27994a19bade84f482d133","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b27994a19bade84f482d133: start","job_status":"start","queue_duration_s":1.289734,"scheduling_latency_s":1.289748}2530gitlab # [ 253.338652] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b27994a19bade84f482d133","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b27994a19bade84f482d133: done: 0.009959 sec","job_status":"done","queue_duration_s":1.289734,"scheduling_latency_s":1.289748,"redis_calls":13,"redis_duration_s":0.002474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009959,"completed_at":"2026-09-03T10:17:12.217Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2531gitlab # [ 253.378110] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"527f8d4c995ad91511bae997","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-527f8d4c995ad91511bae997: start","job_status":"start","queue_duration_s":1.300782,"scheduling_latency_s":1.300796}2532gitlab # [ 253.378780] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"527f8d4c995ad91511bae997","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-527f8d4c995ad91511bae997: done: 0.009973 sec","job_status":"done","queue_duration_s":1.300782,"scheduling_latency_s":1.300796,"redis_calls":13,"redis_duration_s":0.002464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001699,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009973,"completed_at":"2026-09-03T10:17:12.228Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2533gitlab # [ 253.400956] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2713cacfbb7719bb4db4e7e","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2713cacfbb7719bb4db4e7e: start","job_status":"start","queue_duration_s":1.311795,"scheduling_latency_s":1.311807}2534gitlab # [ 253.442309] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2713cacfbb7719bb4db4e7e","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2713cacfbb7719bb4db4e7e: done: 0.010016 sec","job_status":"done","queue_duration_s":1.311795,"scheduling_latency_s":1.311807,"redis_calls":13,"redis_duration_s":0.002539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001672,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010016,"completed_at":"2026-09-03T10:17:12.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2535gitlab # [ 253.446029] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5126b6b9b7c8361ab9cb517","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5126b6b9b7c8361ab9cb517: start","job_status":"start","queue_duration_s":1.322991,"scheduling_latency_s":1.323004}2536gitlab # [ 253.493793] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5126b6b9b7c8361ab9cb517","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5126b6b9b7c8361ab9cb517: done: 0.010411 sec","job_status":"done","queue_duration_s":1.322991,"scheduling_latency_s":1.323004,"redis_calls":13,"redis_duration_s":0.002868,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001769,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010411,"completed_at":"2026-09-03T10:17:12.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2537gitlab # [ 253.494984] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39b30dd07c82718ca1af6188","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39b30dd07c82718ca1af6188: start","job_status":"start","queue_duration_s":1.334557,"scheduling_latency_s":1.33457}2538gitlab # [ 253.532770] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39b30dd07c82718ca1af6188","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39b30dd07c82718ca1af6188: done: 0.009667 sec","job_status":"done","queue_duration_s":1.334557,"scheduling_latency_s":1.33457,"redis_calls":13,"redis_duration_s":0.002042,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001257,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009667,"completed_at":"2026-09-03T10:17:12.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2539gitlab # [ 253.536704] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6272382ee7bb0a8d952d589","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6272382ee7bb0a8d952d589: start","job_status":"start","queue_duration_s":1.34506,"scheduling_latency_s":1.345073}2540gitlab # [ 253.547762] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6272382ee7bb0a8d952d589","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6272382ee7bb0a8d952d589: done: 0.00926 sec","job_status":"done","queue_duration_s":1.34506,"scheduling_latency_s":1.345073,"redis_calls":13,"redis_duration_s":0.0015869999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00926,"completed_at":"2026-09-03T10:17:12.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2541gitlab # [ 253.591421] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d6ded9b9b8b2bd57ab386fc","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d6ded9b9b8b2bd57ab386fc: start","job_status":"start","queue_duration_s":1.355163,"scheduling_latency_s":1.355176}2542gitlab # [ 253.593118] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d6ded9b9b8b2bd57ab386fc","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d6ded9b9b8b2bd57ab386fc: done: 0.009081 sec","job_status":"done","queue_duration_s":1.355163,"scheduling_latency_s":1.355176,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009081,"completed_at":"2026-09-03T10:17:12.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2543gitlab # [ 253.633168] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a52f67d0bae38e96a85fc86","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a52f67d0bae38e96a85fc86: start","job_status":"start","queue_duration_s":1.365108,"scheduling_latency_s":1.365122}2544gitlab # [ 253.638309] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a52f67d0bae38e96a85fc86","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a52f67d0bae38e96a85fc86: done: 0.010028 sec","job_status":"done","queue_duration_s":1.365108,"scheduling_latency_s":1.365122,"redis_calls":13,"redis_duration_s":0.0016950000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010028,"completed_at":"2026-09-03T10:17:12.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2545gitlab # [ 253.682298] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20f92d05d2d1588c71593df1","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20f92d05d2d1588c71593df1: start","job_status":"start","queue_duration_s":1.376043,"scheduling_latency_s":1.376057}2546gitlab # [ 253.683566] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20f92d05d2d1588c71593df1","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20f92d05d2d1588c71593df1: done: 0.009132 sec","job_status":"done","queue_duration_s":1.376043,"scheduling_latency_s":1.376057,"redis_calls":13,"redis_duration_s":0.0015860000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009132,"completed_at":"2026-09-03T10:17:12.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2547gitlab # [ 253.731572] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0af8035522e11c1332955cb","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0af8035522e11c1332955cb: start","job_status":"start","queue_duration_s":1.386031,"scheduling_latency_s":1.386044}2548gitlab # [ 253.732282] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0af8035522e11c1332955cb","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0af8035522e11c1332955cb: done: 0.009128 sec","job_status":"done","queue_duration_s":1.386031,"scheduling_latency_s":1.386044,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009128,"completed_at":"2026-09-03T10:17:12.312Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2549gitlab # [ 253.781098] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf5f81ebab8b4028b3f0144a","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf5f81ebab8b4028b3f0144a: start","job_status":"start","queue_duration_s":1.396022,"scheduling_latency_s":1.396036}2550gitlab # [ 253.781810] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf5f81ebab8b4028b3f0144a","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf5f81ebab8b4028b3f0144a: done: 0.009109 sec","job_status":"done","queue_duration_s":1.396022,"scheduling_latency_s":1.396036,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009109,"completed_at":"2026-09-03T10:17:12.322Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2551gitlab # [ 253.823592] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26ccf4297297592740e43bfa","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26ccf4297297592740e43bfa: start","job_status":"start","queue_duration_s":1.405993,"scheduling_latency_s":1.406005}2552gitlab # [ 253.824269] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26ccf4297297592740e43bfa","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26ccf4297297592740e43bfa: done: 0.00918 sec","job_status":"done","queue_duration_s":1.405993,"scheduling_latency_s":1.406005,"redis_calls":13,"redis_duration_s":0.0016489999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00918,"completed_at":"2026-09-03T10:17:12.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2553gitlab # [ 253.869502] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e48dc8c8588f0c38727d66f0","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e48dc8c8588f0c38727d66f0: start","job_status":"start","queue_duration_s":1.416058,"scheduling_latency_s":1.416071}2554gitlab # [ 253.870133] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e48dc8c8588f0c38727d66f0","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e48dc8c8588f0c38727d66f0: done: 0.009067 sec","job_status":"done","queue_duration_s":1.416058,"scheduling_latency_s":1.416071,"redis_calls":13,"redis_duration_s":0.0016419999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009067,"completed_at":"2026-09-03T10:17:12.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2555gitlab # [ 253.912226] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e80e3ce11b94068fa7de786c","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e80e3ce11b94068fa7de786c: start","job_status":"start","queue_duration_s":1.425983,"scheduling_latency_s":1.425997}2556gitlab # [ 253.912865] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e80e3ce11b94068fa7de786c","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e80e3ce11b94068fa7de786c: done: 0.009248 sec","job_status":"done","queue_duration_s":1.425983,"scheduling_latency_s":1.425997,"redis_calls":13,"redis_duration_s":0.001656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009248,"completed_at":"2026-09-03T10:17:12.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2557gitlab # [ 253.951113] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4cbca5c11e3db02c21fd2a8","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4cbca5c11e3db02c21fd2a8: start","job_status":"start","queue_duration_s":1.436079,"scheduling_latency_s":1.436093}2558gitlab # [ 253.953736] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4cbca5c11e3db02c21fd2a8","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4cbca5c11e3db02c21fd2a8: done: 0.011838 sec","job_status":"done","queue_duration_s":1.436079,"scheduling_latency_s":1.436093,"redis_calls":13,"redis_duration_s":0.0019199999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001248,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011838,"completed_at":"2026-09-03T10:17:12.365Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2559gitlab # [ 254.000145] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65b66a11e607d99b2a186997","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65b66a11e607d99b2a186997: start","job_status":"start","queue_duration_s":1.449012,"scheduling_latency_s":1.449033}2560gitlab # [ 254.000941] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65b66a11e607d99b2a186997","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65b66a11e607d99b2a186997: done: 0.009729 sec","job_status":"done","queue_duration_s":1.449012,"scheduling_latency_s":1.449033,"redis_calls":13,"redis_duration_s":0.001569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009729,"completed_at":"2026-09-03T10:17:12.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2561gitlab # [ 254.034467] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eab219bd310cfb6f92c738ce","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eab219bd310cfb6f92c738ce: start","job_status":"start","queue_duration_s":1.459577,"scheduling_latency_s":1.459591}2562gitlab # [ 254.080510] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eab219bd310cfb6f92c738ce","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eab219bd310cfb6f92c738ce: done: 0.009138 sec","job_status":"done","queue_duration_s":1.459577,"scheduling_latency_s":1.459591,"redis_calls":13,"redis_duration_s":0.001619,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009138,"completed_at":"2026-09-03T10:17:12.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2563gitlab # [ 254.081702] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22b7669cd819070cd1df731d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22b7669cd819070cd1df731d: start","job_status":"start","queue_duration_s":1.469562,"scheduling_latency_s":1.469575}2564gitlab # [ 254.091133] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22b7669cd819070cd1df731d","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22b7669cd819070cd1df731d: done: 0.011615 sec","job_status":"done","queue_duration_s":1.469562,"scheduling_latency_s":1.469575,"redis_calls":13,"redis_duration_s":0.003921,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003383,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011615,"completed_at":"2026-09-03T10:17:12.398Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2565gitlab # [ 254.162144] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9f68757fd64f4c764780504","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9f68757fd64f4c764780504: start","job_status":"start","queue_duration_s":1.482016,"scheduling_latency_s":1.48203}2566gitlab # [ 254.162897] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9f68757fd64f4c764780504","created_at":"2026-09-03T10:17:10.620Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-9a230a698aa44a87","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ce8ae496520425e4f73ec013d84b80cc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.917Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9f68757fd64f4c764780504: done: 0.009092 sec","job_status":"done","queue_duration_s":1.482016,"scheduling_latency_s":1.48203,"redis_calls":13,"redis_duration_s":0.0015669999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009092,"completed_at":"2026-09-03T10:17:12.408Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2567gitlab # [ 254.172046] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.409Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f419bfe71885c0f4afbcdae4","created_at":"2026-09-03T10:17:10.908Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa2168fbd3a7c8ffdd577a06b10a0a8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.919Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f419bfe71885c0f4afbcdae4: start","job_status":"start","queue_duration_s":1.490257,"scheduling_latency_s":1.490271}2568gitlab # [ 254.185566] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.418Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2569gitlab # [ 254.188424] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f419bfe71885c0f4afbcdae4","created_at":"2026-09-03T10:17:10.908Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa2168fbd3a7c8ffdd577a06b10a0a8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.919Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f419bfe71885c0f4afbcdae4: done: 0.01338 sec","job_status":"done","queue_duration_s":1.490257,"scheduling_latency_s":1.490271,"redis_calls":9,"redis_duration_s":0.001437,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000314,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01338,"completed_at":"2026-09-03T10:17:12.422Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000447,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2570gitlab # [ 254.244083] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:12.424Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3c7566f5d9c06e585245a2c2","created_at":"2026-09-03T10:17:10.932Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68710969d248e9009ca8b133b087f6a1","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.022Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3c7566f5d9c06e585245a2c2: start","job_status":"start","queue_duration_s":1.401148,"scheduling_latency_s":1.401167}2571gitlab # [ 254.285688] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 18482572gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses, in 1.99 seconds)2573gitlab # [ 254.335778] gitaly[920]: time="2026-09-03T10:17:14.324Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:44.323" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:14.323" grpc.time_ms=0.488 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2574gitlab # [ 254.362473] gitaly[920]: time="2026-09-03T10:17:14.351Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:24.286" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:14.346" grpc.time_ms=4.819 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2575gitlab # [ 254.406975] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 17882576gitlab # [ 254.408198] nixbot[2597]: {"ts": "2026-09-03T10:17:14.396Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eba459b7c8f87c78f7a7240ca13a81922b9f7662 \"HTTP/1.1 201 Created\""}2577gitlab # [ 254.446579] gitaly[920]: time="2026-09-03T10:17:14.435Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBKD1BCZ0BFZAJ5FZQ35R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:44.434" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:14.434" grpc.time_ms=0.494 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2578gitlab # [ 254.452581] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3c7566f5d9c06e585245a2c2","created_at":"2026-09-03T10:17:10.932Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68710969d248e9009ca8b133b087f6a1","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.022Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3c7566f5d9c06e585245a2c2: done: 2.017294 sec","job_status":"done","queue_duration_s":1.401148,"scheduling_latency_s":1.401167,"redis_calls":565,"redis_duration_s":1.8434979999999999,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":558,"redis_shared_state_duration_s":1.842447,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.188702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":2.017294,"completed_at":"2026-09-03T10:17:14.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2579gitlab # [ 254.498861] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4cab375001123551a69c6ec3","created_at":"2026-09-03T10:17:11.052Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"919a5773d96fc9505052c6c66aaaaae1","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.070Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4cab375001123551a69c6ec3: start","job_status":"start","queue_duration_s":3.372189,"scheduling_latency_s":3.37221}2580gitlab # [ 254.499697] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.456Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4cab375001123551a69c6ec3","created_at":"2026-09-03T10:17:11.052Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"919a5773d96fc9505052c6c66aaaaae1","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.070Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4cab375001123551a69c6ec3: done: 0.014095 sec","job_status":"done","queue_duration_s":3.372189,"scheduling_latency_s":3.37221,"redis_calls":11,"redis_duration_s":0.001683,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000344178,"exclusive_lock_hold_duration_s":0.001683733,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.014095,"completed_at":"2026-09-03T10:17:14.456Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2581gitlab # [ 254.516721] gitaly[920]: time="2026-09-03T10:17:14.462Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KCBKD1BCZ0BFZAJ5FZQ35R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:24.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:14.457" grpc.time_ms=4.677 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2582gitlab # [ 254.528903] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8c1f41182019349122a7fb91","created_at":"2026-09-03T10:17:11.093Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f35351b7836739fadc8459dfa6188fa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:17:11.137Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8c1f41182019349122a7fb91: start","job_status":"start","queue_duration_s":3.31983,"scheduling_latency_s":3.319843}2583gitlab # [ 254.576544] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.464Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2584gitlab # [ 254.577083] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8c1f41182019349122a7fb91","created_at":"2026-09-03T10:17:11.093Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f35351b7836739fadc8459dfa6188fa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:17:11.137Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8c1f41182019349122a7fb91: done: 0.010685 sec","job_status":"done","queue_duration_s":3.31983,"scheduling_latency_s":3.319843,"redis_calls":9,"redis_duration_s":0.001197,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010685,"completed_at":"2026-09-03T10:17:14.467Z","load_balancing_strategy":"primary","db_duration_s":0.000242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2585gitlab # [ 254.580257] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"928fa7a0b2757c2b2e62df53","created_at":"2026-09-03T10:17:11.161Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f263ec938926d47b5c6774ee34daa09","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.176Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-928fa7a0b2757c2b2e62df53: start","job_status":"start","queue_duration_s":3.292179,"scheduling_latency_s":3.292192}2586gitlab # [ 254.620027] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.482Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"928fa7a0b2757c2b2e62df53","created_at":"2026-09-03T10:17:11.161Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f263ec938926d47b5c6774ee34daa09","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.176Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-928fa7a0b2757c2b2e62df53: done: 0.013939 sec","job_status":"done","queue_duration_s":3.292179,"scheduling_latency_s":3.292192,"redis_calls":11,"redis_duration_s":0.001373,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000281321,"exclusive_lock_hold_duration_s":0.003464127,"duration_s":0.013939,"completed_at":"2026-09-03T10:17:14.482Z","load_balancing_strategy":"primary","db_duration_s":0.00157,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2587gitlab # [ 254.628803] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.483Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"d17e72e7923d0cf7603a74dd","created_at":"2026-09-03T10:17:11.191Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9eeb6637b1e63dbb4c8396a9fe3effcb","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.265Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-d17e72e7923d0cf7603a74dd: start","job_status":"start","queue_duration_s":3.217806,"scheduling_latency_s":3.217818}2588gitlab # [ 254.641128] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.499Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"d17e72e7923d0cf7603a74dd","created_at":"2026-09-03T10:17:11.191Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9eeb6637b1e63dbb4c8396a9fe3effcb","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.265Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-d17e72e7923d0cf7603a74dd: done: 0.015874 sec","job_status":"done","queue_duration_s":3.217806,"scheduling_latency_s":3.217818,"redis_calls":12,"redis_duration_s":0.0015630000000000002,"redis_read_bytes":11,"redis_write_bytes":1354,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":297,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000568,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":519,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000286349,"exclusive_lock_hold_duration_s":0.0052780449999999994,"duration_s":0.015874,"completed_at":"2026-09-03T10:17:14.499Z","load_balancing_strategy":"primary","db_duration_s":0.00218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2589gitlab # [ 254.711440] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.500Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"dc340a6621ba4b0a26b1c868","created_at":"2026-09-03T10:17:11.301Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f451117c394c1275f230340dadd04898","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.312Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-dc340a6621ba4b0a26b1c868: start","job_status":"start","queue_duration_s":3.188144,"scheduling_latency_s":3.188156}2590gitlab # [ 254.712177] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"dc340a6621ba4b0a26b1c868","created_at":"2026-09-03T10:17:11.301Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f451117c394c1275f230340dadd04898","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.312Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-dc340a6621ba4b0a26b1c868: done: 0.010398 sec","job_status":"done","queue_duration_s":3.188144,"scheduling_latency_s":3.188156,"redis_calls":9,"redis_duration_s":0.001123,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010398,"completed_at":"2026-09-03T10:17:14.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2591gitlab # [ 254.756274] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.511Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9e563af76c5683880bd2b0a9","created_at":"2026-09-03T10:17:11.327Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.336Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9e563af76c5683880bd2b0a9: start","job_status":"start","queue_duration_s":3.17578,"scheduling_latency_s":3.175794}2592gitlab # [ 254.756997] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9e563af76c5683880bd2b0a9","created_at":"2026-09-03T10:17:11.327Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.336Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9e563af76c5683880bd2b0a9: done: 0.016743 sec","job_status":"done","queue_duration_s":3.17578,"scheduling_latency_s":3.175794,"redis_calls":16,"redis_duration_s":0.00191,"redis_read_bytes":17,"redis_write_bytes":4347,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000237,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_calls":2,"redis_queues_duration_s":0.000176,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001134,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000363,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016743,"completed_at":"2026-09-03T10:17:14.528Z","load_balancing_strategy":"primary","db_duration_s":0.000676,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2593gitlab # [ 254.796153] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9f6088fe62d52d86536a9079","created_at":"2026-09-03T10:17:11.349Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2dd79ae2d48f8d4fc1faceb2b7747a73","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.415Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9f6088fe62d52d86536a9079: start","job_status":"start","queue_duration_s":3.114406,"scheduling_latency_s":3.114419}2594gitlab # [ 254.796887] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9f6088fe62d52d86536a9079","created_at":"2026-09-03T10:17:11.349Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2dd79ae2d48f8d4fc1faceb2b7747a73","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.415Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9f6088fe62d52d86536a9079: done: 0.172631 sec","job_status":"done","queue_duration_s":3.114406,"scheduling_latency_s":3.114419,"redis_calls":11,"redis_duration_s":0.0018089999999999998,"redis_read_bytes":9,"redis_write_bytes":1292,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000282,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.147156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000360101,"exclusive_lock_hold_duration_s":0.104521943,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.172631,"completed_at":"2026-09-03T10:17:14.702Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027603,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2595gitlab # [ 254.832662] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6735e64ed681687f0f2dbe9f","created_at":"2026-09-03T10:17:11.429Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62a4ba875b06a44cf618db0bad34ce5d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.445Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6735e64ed681687f0f2dbe9f: start","job_status":"start","queue_duration_s":3.258103,"scheduling_latency_s":3.258119}2596gitlab # [ 254.845683] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.721Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6735e64ed681687f0f2dbe9f","created_at":"2026-09-03T10:17:11.429Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62a4ba875b06a44cf618db0bad34ce5d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.445Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6735e64ed681687f0f2dbe9f: done: 0.018659 sec","job_status":"done","queue_duration_s":3.258103,"scheduling_latency_s":3.258119,"redis_calls":9,"redis_duration_s":0.001224,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000696,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000243,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.018659,"completed_at":"2026-09-03T10:17:14.721Z","load_balancing_strategy":"primary","db_duration_s":0.004922,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2597gitlab # [ 254.885643] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.722Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fc36d60cc7c22d3bc380f5e5","created_at":"2026-09-03T10:17:11.468Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94aaef2c5651a33c0c549e4ca0e1402f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:17:11.520Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fc36d60cc7c22d3bc380f5e5: start","job_status":"start","queue_duration_s":3.202396,"scheduling_latency_s":3.20241}2598gitlab # [ 254.932813] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.734Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fc36d60cc7c22d3bc380f5e5","created_at":"2026-09-03T10:17:11.468Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94aaef2c5651a33c0c549e4ca0e1402f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:17:11.520Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fc36d60cc7c22d3bc380f5e5: done: 0.01214 sec","job_status":"done","queue_duration_s":3.202396,"scheduling_latency_s":3.20241,"redis_calls":7,"redis_duration_s":0.0008669999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000633,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01214,"completed_at":"2026-09-03T10:17:14.734Z","load_balancing_strategy":"primary","db_duration_s":0.000477,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2599gitlab # [ 254.933958] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.735Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"37382427cc87ac8e1345f3ec","created_at":"2026-09-03T10:17:11.547Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b88f23bde0421a893ec5a4012f274ccb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.554Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-37382427cc87ac8e1345f3ec: start","job_status":"start","queue_duration_s":3.181268,"scheduling_latency_s":3.181282}2600gitlab # [ 254.944123] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.740Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2601gitlab # [ 254.984213] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.743Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"37382427cc87ac8e1345f3ec","created_at":"2026-09-03T10:17:11.547Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-5d8a6ba63f3d41c8","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b88f23bde0421a893ec5a4012f274ccb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.554Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-37382427cc87ac8e1345f3ec: done: 0.007572 sec","job_status":"done","queue_duration_s":3.181268,"scheduling_latency_s":3.181282,"redis_calls":7,"redis_duration_s":0.000829,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007572,"completed_at":"2026-09-03T10:17:14.743Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2602gitlab # [ 254.985203] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:14.744Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d3344062403556fca0e1a974","created_at":"2026-09-03T10:17:14.381Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-af9a70f0f3494c5c","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.384Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d3344062403556fca0e1a974: start","job_status":"start","queue_duration_s":0.359476,"scheduling_latency_s":0.359489}2603gitlab # [ 255.040901] gitaly[920]: time="2026-09-03T10:17:15.029Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:15.029" grpc.time_ms=0.295 pid=920 span.kind=server system=grpc2604gitlab # [ 255.068280] gitaly[920]: time="2026-09-03T10:17:15.057Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:15.056" grpc.time_ms=0.37 pid=920 span.kind=server system=grpc2605gitlab # [ 255.069839] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 6572606gitlab # [ 255.070594] nixbot[2597]: {"ts": "2026-09-03T10:17:15.059Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eba459b7c8f87c78f7a7240ca13a81922b9f7662 \"HTTP/1.1 201 Created\""}2607gitlab # [ 255.098082] gitaly[920]: time="2026-09-03T10:17:15.087Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:15.086" grpc.time_ms=0.213 pid=920 span.kind=server system=grpc2608gitlab # [ 255.108135] gitaly[920]: time="2026-09-03T10:17:15.097Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:44.791" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:14.791" operations="[repack_refs]" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2609gitlab # [ 255.110243] gitaly[920]: time="2026-09-03T10:17:15.099Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:44.791" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:14.791" pid=920 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-03 10:16:08.162905739 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2610gitlab # [ 255.119218] gitaly[920]: time="2026-09-03T10:17:15.108Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBM1VJJH20NJK6MJAN41H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:45.007" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:15.107" grpc.time_ms=0.451 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2611gitlab # [ 255.122881] gitaly[920]: time="2026-09-03T10:17:15.111Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:44.791" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:14.791" pid=920 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2612gitlab # [ 255.123615] gitaly[920]: time="2026-09-03T10:17:15.112Z" level=info msg="optimized repository" clean_stale_data_time=12.305195ms clean_work_trees_time="69.003µs" commit_graph_time="3.632µs" correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="4.191µs" pid=920 prune_time="3.632µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.19µs" storage_name=default system=housekeeping total_time=12.482871ms2613gitlab # [ 255.124029] gitaly[920]: time="2026-09-03T10:17:15.112Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:44.791" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:14.791" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2614gitlab # [ 255.144288] gitaly[920]: time="2026-09-03T10:17:15.133Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1KCBM1VJJH20NJK6MJAN41H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:17:25.128" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:15.128" grpc.time_ms=4.31 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2615gitlab # [ 255.172596] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/eba459b7c8f87c78f7a7240ca13a81922b9f7662 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 932616gitlab # [ 255.173432] nixbot[2597]: {"ts": "2026-09-03T10:17:15.162Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/eba459b7c8f87c78f7a7240ca13a81922b9f7662 \"HTTP/1.1 201 Created\""}2617gitlab # [ 255.186801] gitaly[920]: time="2026-09-03T10:17:15.175Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN diskcache=4f37327b-2e59-4da5-b646-3fe840a55fb5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:44.791" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:14.791" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2618gitlab # [ 255.187467] gitaly[920]: time="2026-09-03T10:17:15.175Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=78 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=823296 command.minflt=8326 command.oublock=0 command.real_time_ms=312 command.spawn_token_fork_ms=0 command.system_time_ms=54 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:44.791" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:14.791" grpc.time_ms=384.312 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2619gitlab # [ 255.228158] gitaly[920]: time="2026-09-03T10:17:15.217Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:17:45.216" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:15.216" grpc.time_ms=0.47 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2620gitlab # [ 255.249194] gitaly[920]: time="2026-09-03T10:17:15.238Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:45.237" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:15.237" grpc.time_ms=0.412 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2621gitlab # [ 255.268966] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.257Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"04f09993d49b43166a74c3cb","created_at":"2026-09-03T10:17:15.256Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"d3344062403556fca0e1a974","job_size_bytes":3,"pid":2556,"job_status":"deduplicated","message":"PipelineProcessWorker JID-04f09993d49b43166a74c3cb: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2622gitlab # [ 255.283277] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.258Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d3344062403556fca0e1a974","created_at":"2026-09-03T10:17:14.381Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-af9a70f0f3494c5c","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.384Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2556,"job_status":"rescheduled","message":"PipelineProcessWorker JID-d3344062403556fca0e1a974: rescheduled"}2623gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses2624gitlab # [ 255.284027] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.264Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d3344062403556fca0e1a974","created_at":"2026-09-03T10:17:14.381Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-af9a70f0f3494c5c","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.384Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d3344062403556fca0e1a974: done: 0.519807 sec","job_status":"done","queue_duration_s":0.359476,"scheduling_latency_s":0.359489,"gitaly_calls":3,"gitaly_duration_s":0.402078,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.009812,"redis_read_bytes":243,"redis_write_bytes":21779,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000581,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000913,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.002468,"redis_queues_read_bytes":32,"redis_queues_write_bytes":14464,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.003329,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000162,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000653,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090087,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.493099821,"duration_s":0.519807,"completed_at":"2026-09-03T10:17:15.264Z","load_balancing_strategy":"primary","db_duration_s":0.023246,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2625gitlab # [ 255.337738] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.265Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0f54f4161befa3c454fac99b","created_at":"2026-09-03T10:17:14.519Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-020e44820a1143f2","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.524Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0f54f4161befa3c454fac99b: start","job_status":"start","queue_duration_s":0.740917,"scheduling_latency_s":0.740945}2626gitlab # [ 255.338514] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.276Z","message":"No existing merge request to be cleaned up."}2627gitlab # [ 255.338906] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.281Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0f54f4161befa3c454fac99b","created_at":"2026-09-03T10:17:14.519Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-020e44820a1143f2","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.524Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0f54f4161befa3c454fac99b: done: 0.016473 sec","job_status":"done","queue_duration_s":0.740917,"scheduling_latency_s":0.740945,"redis_calls":10,"redis_duration_s":0.001443,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016473,"completed_at":"2026-09-03T10:17:15.281Z","load_balancing_strategy":"primary","db_duration_s":0.004901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2628gitlab # [ 255.348441] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.282Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bdd58d4c4b7efdd480563e5b","created_at":"2026-09-03T10:17:14.519Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-020e44820a1143f2","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.524Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bdd58d4c4b7efdd480563e5b: start","job_status":"start","queue_duration_s":0.758231,"scheduling_latency_s":0.758248}2629gitlab # [ 255.359464] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.289Z","message":"No existing merge request to be cleaned up."}2630gitlab # [ 255.360714] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.294Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bdd58d4c4b7efdd480563e5b","created_at":"2026-09-03T10:17:14.519Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-020e44820a1143f2","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.524Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bdd58d4c4b7efdd480563e5b: done: 0.011546 sec","job_status":"done","queue_duration_s":0.758231,"scheduling_latency_s":0.758248,"redis_calls":10,"redis_duration_s":0.001273,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011546,"completed_at":"2026-09-03T10:17:15.294Z","load_balancing_strategy":"primary","db_duration_s":0.000945,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2631gitlab # [ 255.403166] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.295Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2388ba2e7c15ab9be65146f2","created_at":"2026-09-03T10:17:14.519Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-020e44820a1143f2","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.524Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2388ba2e7c15ab9be65146f2: start","job_status":"start","queue_duration_s":0.770588,"scheduling_latency_s":0.770602}2632gitlab # [ 255.406974] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.301Z","message":"No existing merge request to be cleaned up."}2633gitlab # [ 255.446089] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.305Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2388ba2e7c15ab9be65146f2","created_at":"2026-09-03T10:17:14.519Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-020e44820a1143f2","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.524Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2388ba2e7c15ab9be65146f2: done: 0.010721 sec","job_status":"done","queue_duration_s":0.770588,"scheduling_latency_s":0.770602,"redis_calls":10,"redis_duration_s":0.001277,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010721,"completed_at":"2026-09-03T10:17:15.305Z","load_balancing_strategy":"primary","db_duration_s":0.001039,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2634gitlab # [ 255.446938] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.306Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a6c2e93520ac5c0fff7c7f56","created_at":"2026-09-03T10:17:14.519Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-020e44820a1143f2","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.524Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a6c2e93520ac5c0fff7c7f56: start","job_status":"start","queue_duration_s":0.782136,"scheduling_latency_s":0.78215}2635gitlab # [ 255.487788] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.312Z","message":"No existing merge request to be cleaned up."}2636gitlab # [ 255.488600] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 872637gitlab # [ 255.489046] gitlab-workhorse[826]: time="2026-09-03T10:17:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672638gitlab # [ 255.489546] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 432639gitlab # [ 255.490301] gitaly[920]: time="2026-09-03T10:17:15.359Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCBM88MM76JVKED7WSV4QT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:15.353" grpc.time_ms=5.644 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2640gitlab # [ 255.490859] gitaly[920]: time="2026-09-03T10:17:15.404Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBMB1T0W5ZZBSHT2P0WCT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:15.398" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2641gitlab # [ 255.491561] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.317Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a6c2e93520ac5c0fff7c7f56","created_at":"2026-09-03T10:17:14.519Z","trace_propagation_headers":{"sentry-trace":"4123d6bdbd2347fda45292b8bf98fe07-020e44820a1143f2","baggage":"sentry-trace_id=4123d6bdbd2347fda45292b8bf98fe07,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7762755cf5097caeb659495e84edb31e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.524Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a6c2e93520ac5c0fff7c7f56: done: 0.010331 sec","job_status":"done","queue_duration_s":0.782136,"scheduling_latency_s":0.78215,"redis_calls":10,"redis_duration_s":0.001313,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010331,"completed_at":"2026-09-03T10:17:15.317Z","load_balancing_strategy":"primary","db_duration_s":0.000892,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2642gitlab # [ 255.535817] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.318Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a9164bf5c1dd3765828caea2","created_at":"2026-09-03T10:17:14.786Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.788Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a9164bf5c1dd3765828caea2: start","job_status":"start","queue_duration_s":0.530197,"scheduling_latency_s":0.530211}2643gitlab # [ 255.536783] gitaly[920]: time="2026-09-03T10:17:15.404Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBMB1T0W5ZZBSHT2P0WCT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:15.398" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2644gitlab # [ 255.543601] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.342Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a9164bf5c1dd3765828caea2","created_at":"2026-09-03T10:17:14.786Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:14.788Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a9164bf5c1dd3765828caea2: done: 0.02408 sec","job_status":"done","queue_duration_s":0.530197,"scheduling_latency_s":0.530211,"redis_calls":7,"redis_duration_s":0.000935,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02408,"completed_at":"2026-09-03T10:17:15.342Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004628,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2645gitlab # [ 255.606143] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.343Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a28f7a0921dce4f49303117d","created_at":"2026-09-03T10:17:15.180Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.182Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a28f7a0921dce4f49303117d: start","job_status":"start","queue_duration_s":0.160772,"scheduling_latency_s":0.160785}2646gitlab # [ 255.606967] gitaly[920]: time="2026-09-03T10:17:15.405Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBMB1T0W5ZZBSHT2P0WCT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:17:15.398" grpc.time_ms=5.98 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2647gitlab # [ 255.611174] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.359Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a28f7a0921dce4f49303117d","created_at":"2026-09-03T10:17:15.180Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.182Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a28f7a0921dce4f49303117d: done: 0.015893 sec","job_status":"done","queue_duration_s":0.160772,"scheduling_latency_s":0.160785,"redis_calls":7,"redis_duration_s":0.000919,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015893,"completed_at":"2026-09-03T10:17:15.359Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002547,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2648gitlab # [ 255.661479] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.360Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"0a1a2826c3e0742a6fc9ed88","created_at":"2026-09-03T10:17:15.187Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.188Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-0a1a2826c3e0742a6fc9ed88: start","job_status":"start","queue_duration_s":0.171592,"scheduling_latency_s":0.171606}2649gitlab # [ 255.702329] gitaly[920]: time="2026-09-03T10:17:15.573Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:15.573" grpc.time_ms=0.304 pid=920 span.kind=server system=grpc2650gitlab # [ 255.702707] gitaly[920]: time="2026-09-03T10:17:15.593Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:15.592" grpc.time_ms=0.272 pid=920 span.kind=server system=grpc2651gitlab # [ 255.703961] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.379Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"0a1a2826c3e0742a6fc9ed88","created_at":"2026-09-03T10:17:15.187Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.188Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-0a1a2826c3e0742a6fc9ed88: done: 0.019685 sec","job_status":"done","queue_duration_s":0.171592,"scheduling_latency_s":0.171606,"redis_calls":7,"redis_duration_s":0.000909,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019685,"completed_at":"2026-09-03T10:17:15.379Z","load_balancing_strategy":"primary","db_duration_s":0.005823,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2652gitlab # [ 255.720972] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.380Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"f7000eb8ccb37c1fde828b4c","created_at":"2026-09-03T10:17:15.194Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.195Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f7000eb8ccb37c1fde828b4c: start","job_status":"start","queue_duration_s":0.185231,"scheduling_latency_s":0.185245}2653gitlab # [ 255.735527] gitaly[920]: time="2026-09-03T10:17:15.611Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:17:15.611" grpc.time_ms=0.318 pid=920 span.kind=server system=grpc2654gitlab # [ 255.744113] gitaly[920]: time="2026-09-03T10:17:15.619Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN diskcache=91f62771-1230-4f81-8569-aa5a2e698166 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:45.553" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:15.553" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2655gitlab # [ 255.782735] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.402Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"f7000eb8ccb37c1fde828b4c","created_at":"2026-09-03T10:17:15.194Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.195Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f7000eb8ccb37c1fde828b4c: done: 0.021759 sec","job_status":"done","queue_duration_s":0.185231,"scheduling_latency_s":0.185245,"redis_calls":10,"redis_duration_s":0.00138,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000409,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000277,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021759,"completed_at":"2026-09-03T10:17:15.402Z","load_balancing_strategy":"primary","db_duration_s":0.004279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2656gitlab # [ 255.791601] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.403Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"20b7ca53dedafe0b95bcbaed","created_at":"2026-09-03T10:17:15.195Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.196Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-20b7ca53dedafe0b95bcbaed: start","job_status":"start","queue_duration_s":0.206698,"scheduling_latency_s":0.206714}2657gitlab # [ 255.839734] gitaly[920]: time="2026-09-03T10:17:15.620Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=66 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7709 command.oublock=0 command.real_time_ms=60 command.spawn_token_fork_ms=1 command.system_time_ms=35 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:17:45.553" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:17:15.553" grpc.time_ms=66.557 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2658gitlab # [ 255.840611] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.421Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"20b7ca53dedafe0b95bcbaed","created_at":"2026-09-03T10:17:15.195Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.196Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-20b7ca53dedafe0b95bcbaed: done: 0.017771 sec","job_status":"done","queue_duration_s":0.206698,"scheduling_latency_s":0.206714,"redis_calls":7,"redis_duration_s":0.001022,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017771,"completed_at":"2026-09-03T10:17:15.420Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2659gitlab # [ 255.856696] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.422Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7cce7ed710f9158f52d04394","created_at":"2026-09-03T10:17:15.196Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.198Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7cce7ed710f9158f52d04394: start","job_status":"start","queue_duration_s":0.223995,"scheduling_latency_s":0.224008}2660gitlab # [ 255.896610] gitaly[920]: time="2026-09-03T10:17:15.657Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:17:45.656" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:15.656" grpc.time_ms=0.602 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2661gitlab # [ 255.897181] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.437Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7cce7ed710f9158f52d04394","created_at":"2026-09-03T10:17:15.196Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.198Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7cce7ed710f9158f52d04394: done: 0.015201 sec","job_status":"done","queue_duration_s":0.223995,"scheduling_latency_s":0.224008,"redis_calls":8,"redis_duration_s":0.001214,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000277,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015201,"completed_at":"2026-09-03T10:17:15.437Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001832,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2662gitlab # [ 255.944368] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.438Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d90f1eeceb201b64af619490","created_at":"2026-09-03T10:17:15.199Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.201Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d90f1eeceb201b64af619490: start","job_status":"start","queue_duration_s":0.236769,"scheduling_latency_s":0.236783}2663gitlab # [ 255.945079] gitaly[920]: time="2026-09-03T10:17:15.676Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCBHMZXJSRT6KSJ9PZJ2EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:17:45.675" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:17:15.676" grpc.time_ms=0.665 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2664gitlab # [ 255.945747] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.451Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d90f1eeceb201b64af619490","created_at":"2026-09-03T10:17:15.199Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.201Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d90f1eeceb201b64af619490: done: 0.01303 sec","job_status":"done","queue_duration_s":0.236769,"scheduling_latency_s":0.236783,"redis_calls":7,"redis_duration_s":0.000908,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01303,"completed_at":"2026-09-03T10:17:15.451Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2665gitlab # [ 256.019903] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.452Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"474f74c7e387e4a27aae2fe3","created_at":"2026-09-03T10:17:15.202Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.203Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-474f74c7e387e4a27aae2fe3: start","job_status":"start","queue_duration_s":0.248937,"scheduling_latency_s":0.24895}2666gitlab # [ 256.020792] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.475Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"474f74c7e387e4a27aae2fe3","created_at":"2026-09-03T10:17:15.202Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.203Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-474f74c7e387e4a27aae2fe3: done: 0.023262 sec","job_status":"done","queue_duration_s":0.248937,"scheduling_latency_s":0.24895,"redis_calls":7,"redis_duration_s":0.002407,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023262,"completed_at":"2026-09-03T10:17:15.475Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2667gitlab # [ 256.036710] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.476Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"3edebba441377dd4cb79bfb3","created_at":"2026-09-03T10:17:15.206Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.207Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3edebba441377dd4cb79bfb3: start","job_status":"start","queue_duration_s":0.269458,"scheduling_latency_s":0.269473}2668gitlab # [ 256.051755] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.491Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"3edebba441377dd4cb79bfb3","created_at":"2026-09-03T10:17:15.206Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.207Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3edebba441377dd4cb79bfb3: done: 0.015281 sec","job_status":"done","queue_duration_s":0.269458,"scheduling_latency_s":0.269473,"redis_calls":7,"redis_duration_s":0.000919,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015281,"completed_at":"2026-09-03T10:17:15.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2669gitlab # [ 256.090168] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.492Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"cfcee3355ec82573629274fd","created_at":"2026-09-03T10:17:15.207Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.208Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-cfcee3355ec82573629274fd: start","job_status":"start","queue_duration_s":0.284375,"scheduling_latency_s":0.284388}2670gitlab # [ 256.128674] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.500Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"cfcee3355ec82573629274fd","created_at":"2026-09-03T10:17:15.207Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.208Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-cfcee3355ec82573629274fd: done: 0.008295 sec","job_status":"done","queue_duration_s":0.284375,"scheduling_latency_s":0.284388,"redis_calls":7,"redis_duration_s":0.000989,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008295,"completed_at":"2026-09-03T10:17:15.500Z","load_balancing_strategy":"primary","db_duration_s":0.000327,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2671gitlab # [ 256.169099] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.501Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"0b98103cb3de14d195a0dd83","created_at":"2026-09-03T10:17:15.208Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.209Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0b98103cb3de14d195a0dd83: start","job_status":"start","queue_duration_s":0.291876,"scheduling_latency_s":0.291888}2672gitlab # [ 256.169814] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.514Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"0b98103cb3de14d195a0dd83","created_at":"2026-09-03T10:17:15.208Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.209Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0b98103cb3de14d195a0dd83: done: 0.01293 sec","job_status":"done","queue_duration_s":0.291876,"scheduling_latency_s":0.291888,"redis_calls":7,"redis_duration_s":0.000838,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01293,"completed_at":"2026-09-03T10:17:15.514Z","load_balancing_strategy":"primary","db_duration_s":0.001804,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2673gitlab # [ 256.219839] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.515Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"5c4ad8b73112b545c199a177","created_at":"2026-09-03T10:17:15.259Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.259Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5c4ad8b73112b545c199a177: start","job_status":"start","queue_duration_s":0.255507,"scheduling_latency_s":0.255519}2674gitlab # [ 256.220723] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.684Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"5c4ad8b73112b545c199a177","created_at":"2026-09-03T10:17:15.259Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-ac4b7c8323c14ea0","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.259Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5c4ad8b73112b545c199a177: done: 0.169107 sec","job_status":"done","queue_duration_s":0.255507,"scheduling_latency_s":0.255519,"gitaly_calls":3,"gitaly_duration_s":0.073476,"redis_calls":65,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008118,"redis_read_bytes":144,"redis_write_bytes":16876,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000329,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000903,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":20,"redis_queues_duration_s":0.001919,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11225,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.00293,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000154,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000544,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074755,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.150606572,"duration_s":0.169107,"completed_at":"2026-09-03T10:17:15.684Z","load_balancing_strategy":"primary","db_duration_s":0.017594,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2675gitlab # [ 256.259842] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.685Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8078acda4b029a874335cfc2","created_at":"2026-09-03T10:17:15.548Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.550Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8078acda4b029a874335cfc2: start","job_status":"start","queue_duration_s":0.135463,"scheduling_latency_s":0.135478}2676gitlab # [ 256.299781] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.708Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8078acda4b029a874335cfc2","created_at":"2026-09-03T10:17:15.548Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.550Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8078acda4b029a874335cfc2: done: 0.023362 sec","job_status":"done","queue_duration_s":0.135463,"scheduling_latency_s":0.135478,"redis_calls":7,"redis_duration_s":0.000902,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023362,"completed_at":"2026-09-03T10:17:15.708Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004068,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2677gitlab # [ 256.301197] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.709Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"77548868e3e46405354dc401","created_at":"2026-09-03T10:17:15.623Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.625Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-77548868e3e46405354dc401: start","job_status":"start","queue_duration_s":0.084355,"scheduling_latency_s":0.084369}2678gitlab # [ 256.341034] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.725Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"77548868e3e46405354dc401","created_at":"2026-09-03T10:17:15.623Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.625Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-77548868e3e46405354dc401: done: 0.015372 sec","job_status":"done","queue_duration_s":0.084355,"scheduling_latency_s":0.084369,"redis_calls":7,"redis_duration_s":0.0009010000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015372,"completed_at":"2026-09-03T10:17:15.725Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002267,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2679gitlab # [ 256.379433] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.726Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5f96c5e19ab0f71575758168","created_at":"2026-09-03T10:17:15.629Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.632Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5f96c5e19ab0f71575758168: start","job_status":"start","queue_duration_s":0.093929,"scheduling_latency_s":0.093943}2680gitlab # [ 256.380167] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.742Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5f96c5e19ab0f71575758168","created_at":"2026-09-03T10:17:15.629Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.632Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5f96c5e19ab0f71575758168: done: 0.015858 sec","job_status":"done","queue_duration_s":0.093929,"scheduling_latency_s":0.093943,"redis_calls":7,"redis_duration_s":0.000913,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015858,"completed_at":"2026-09-03T10:17:15.742Z","load_balancing_strategy":"primary","db_duration_s":0.002512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2681gitlab # [ 256.438349] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.742Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"9223fa64b5727ccc37d1bccf","created_at":"2026-09-03T10:17:15.637Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.638Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9223fa64b5727ccc37d1bccf: start","job_status":"start","queue_duration_s":0.104494,"scheduling_latency_s":0.104508}2682gitlab # [ 256.439082] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.760Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"9223fa64b5727ccc37d1bccf","created_at":"2026-09-03T10:17:15.637Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.638Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9223fa64b5727ccc37d1bccf: done: 0.017268 sec","job_status":"done","queue_duration_s":0.104494,"scheduling_latency_s":0.104508,"redis_calls":10,"redis_duration_s":0.001242,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000348,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000243,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017268,"completed_at":"2026-09-03T10:17:15.760Z","load_balancing_strategy":"primary","db_duration_s":0.002567,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2683gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/eba459b7c8f87c78f7a7240ca13a81922b9f7662/statuses, in 1.17 seconds)2684(finished: subtest: eval and build post commit statuses, in 8.74 seconds)2685(finished: run the VM test script, in 324.10 seconds)2686gitlab # [ 256.457632] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.761Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b5083e48b7da09bc6677f786","created_at":"2026-09-03T10:17:15.638Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.639Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b5083e48b7da09bc6677f786: start","job_status":"start","queue_duration_s":0.121342,"scheduling_latency_s":0.121355}2687test script finished in 324.12s2688cleanup2689kill QemuMachine (pid 46)2690gitlab # [ 256.505774] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-03T10:17:15.775Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b5083e48b7da09bc6677f786","created_at":"2026-09-03T10:17:15.638Z","trace_propagation_headers":{"sentry-trace":"010349ee8360405ba6635a90e606ff49-848193a9f4dd4511","baggage":"sentry-trace_id=010349ee8360405ba6635a90e606ff49,sentry-release=Unknown"},"correlation_id":"01M1KCBHMZXJSRT6KSJ9PZJ2EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:15.639Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b5083e48b7da09bc6677f786: done: 0.014548 sec","job_status":"done","queue_duration_s":0.121342,"scheduling_latency_s":0.121355,"redis_calls":7,"redis_duration_s":0.000923,"redis_read_bytes":5,"redis_qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2691gitlab # w2692(finished: cleanup, in 1.21 seconds)2693additionally exposed symbols:2694 gitlab,2695 vlan1,2696 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2697[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-03T10:16:12.454Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2698[]2699[]2700[]2701[]2702[]2703[]2704[]2705[]2706[]2707[]2708[]2709[]2710[]2711[]2712[]2713[]2714[]2715[]2716[]2717[]2718[]2719[]2720[]2721[]2722[]2723[]2724[]2725[]2726[]2727[]2728[]2729[]2730[]2731[]2732[]2733[]2734[]2735[]2736[]2737[]2738[]2739[]2740[]2741[]2742[]2743[]2744[]2745[]2746[]2747[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'eba459b7c8f87c78f7a7240ca13a81922b9f7662', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-03T10:17:03.419398Z', 'started_at': '2026-09-03T10:17:03.623457Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2748[{'id': 5, 'sha': 'eba459b7c8f87c78f7a7240ca13a81922b9f7662', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:17:03.601Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2749[{'id': 5, 'sha': 'eba459b7c8f87c78f7a7240ca13a81922b9f7662', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:17:03.601Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2750[{'id': 5, 'sha': 'eba459b7c8f87c78f7a7240ca13a81922b9f7662', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:17:03.601Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2751[{'id': 5, 'sha': 'eba459b7c8f87c78f7a7240ca13a81922b9f7662', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:17:03.601Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2752[{'id': 5, 'sha': 'eba459b7c8f87c78f7a7240ca13a81922b9f7662', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:17:03.601Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2753[{'id': 5, 'sha': 'eba459b7c8f87c78f7a7240ca13a81922b9f7662', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 8s (1 warning)', 'created_at': '2026-09-03T10:17:03.601Z', 'started_at': None, 'finished_at': '2026-09-03T10:17:14.375Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'eba459b7c8f87c78f7a7240ca13a81922b9f7662', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-03T10:17:15.044Z', 'started_at': None, 'finished_at': '2026-09-03T10:17:15.146Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]