vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #387
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.2wYbSl5q7A', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: 8f1917b6-b19f-4005-8af5-0990c3c1615816gitlab # Superblock backups stored on blocks:17gitlab # 32768, 98304, 163840, 22937618gitlab # 19gitlab # Allocating group tables: 0/8 done20gitlab # Writing inode tables: 0/8 done21gitlab # Creating journal (8192 blocks): done22gitlab # Writing superblocks and filesystem accounting information: 0/8 done23(finished: start all VMs, in 62.01 seconds)24gitlab # 25gitlab: waiting for unit setup-gitlab.service26gitlab # Virtualisation disk image created.27gitlab: waiting for the VM to finish booting28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/16721jrf13jly83h6234wkpchnp4dgh5-nixos-system-gitlab-test/init regInfo=/nix/store/5a34jx1sa5brxwsi6z9wr1sswqnxw5gh-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 455753353 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor67gitlab # [ 0.000812] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000880] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002740] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002751] Using GB pages for direct mapping73gitlab # [ 0.002851] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002860] ACPI: Early table checksum verification disabled75gitlab # [ 0.002863] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002867] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002871] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002878] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002881] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002882] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002884] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002885] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002887] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002888] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002888] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002889] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002889] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002890] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003122] No NUMA configuration found90gitlab # [ 0.003123] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003126] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]92gitlab # [ 0.005500] Zone ranges:93gitlab # [ 0.005501] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005502] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005503] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005504] Device empty97gitlab # [ 0.005505] Movable zone start for each node98gitlab # [ 0.005506] Early memory node ranges99gitlab # [ 0.005506] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005507] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005508] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005509] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005530] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005805] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.113108] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.113555] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.113570] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.113606] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.113609] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.113611] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.113612] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.113613] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.113614] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.113616] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.113617] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.113622] TSC deadline timer available117gitlab # [ 0.113627] CPU topo: Max. logical packages: 1118gitlab # [ 0.113628] CPU topo: Max. logical dies: 1119gitlab # [ 0.113628] CPU topo: Max. dies per package: 1120gitlab # [ 0.113631] CPU topo: Max. threads per core: 1121gitlab # [ 0.113632] CPU topo: Num. cores per package: 4122gitlab # [ 0.113633] CPU topo: Num. threads per package: 4123gitlab # [ 0.113633] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.113659] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.113678] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.113680] kvm-guest: setup PV sched yield127gitlab # [ 0.113697] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.113698] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.113699] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.113701] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.113702] Booting paravirtualized kernel on KVM132gitlab # [ 0.113706] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.118175] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.120365] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.120416] kvm-guest: PV spinlocks enabled136gitlab # [ 0.120417] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.120420] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/16721jrf13jly83h6234wkpchnp4dgh5-nixos-system-gitlab-test/init regInfo=/nix/store/5a34jx1sa5brxwsi6z9wr1sswqnxw5gh-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.120520] Unknown kernel command line parameters "regInfo=/nix/store/5a34jx1sa5brxwsi6z9wr1sswqnxw5gh-closure-info/registration", will be passed to user space.139gitlab # [ 0.120532] random: crng init done140gitlab # [ 0.120533] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.129130] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.133531] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.133608] software IO TLB: area num 4.144gitlab # [ 0.205951] Fallback order for Node 0: 0145gitlab # [ 0.205960] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.205963] Policy zone: Normal147gitlab # [ 0.208445] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.218348] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.231528] allocated 12582912 bytes of page_ext150gitlab # [ 0.241707] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.241709] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.242586] Dynamic Preempt: lazy153gitlab # [ 0.242769] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.242769] rcu: RCU event tracing is enabled.155gitlab # [ 0.242770] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.242771] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.242772] Rude variant of Tasks RCU enabled.158gitlab # [ 0.242772] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.242773] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.242774] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.242794] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.242796] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.242797] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.247211] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.247574] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.247584] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.247695] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.251323] Console: colour VGA+ 80x25169gitlab # [ 0.251325] printk: legacy console [tty0] enabled170gitlab # [ 0.282438] printk: legacy console [ttyS0] enabled171gitlab # [ 0.390970] ACPI: Core revision 20250807172gitlab # [ 0.391937] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.393551] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.394567] x2apic enabled175gitlab # [ 0.395313] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.396261] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.397536] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.399039] kvm-guest: setup PV IPIs179gitlab # [ 0.400790] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.401826] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns181gitlab # [ 0.403607] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)182gitlab # [ 0.404694] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.405743] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.407604] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.408609] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.409605] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.410605] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.412405] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.413359] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.414417] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.415612] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.416605] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.417605] active return thunk: srso_alias_return_thunk194gitlab # [ 0.418626] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.419605] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.421435] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.422431] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.423604] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.424604] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.425605] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.426605] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.427604] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.428604] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.429604] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.430604] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.431605] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.462461] Freeing SMP alternatives memory: 44K208gitlab # [ 0.463218] pid_max: default: 32768 minimum: 301209gitlab # [ 0.463721] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.464717] landlock: Up and running.211gitlab # [ 0.465605] Yama: becoming mindful.212gitlab # [ 0.466819] LSM support for eBPF active213gitlab # [ 0.467839] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.468749] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.471847] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.473174] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.473613] ... version: 2218gitlab # [ 0.474381] ... bit width: 48219gitlab # [ 0.474607] ... generic counters: 6220gitlab # [ 0.475345] ... generic bitmap: 000000000000003f221gitlab # [ 0.475607] ... fixed-purpose counters: 0222gitlab # [ 0.476367] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.476608] ... value mask: 0000ffffffffffff224gitlab # [ 0.477555] ... max period: 00007fffffffffff225gitlab # [ 0.478350] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.478720] signal: max sigframe size: 3376227gitlab # [ 0.479545] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.480287] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.480811] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.486120] smp: Bringing up secondary CPUs ...231gitlab # [ 0.487479] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.488251] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.490741] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.492607] smpboot: Total of 4 processors activated (19200.09 BogoMIPS)235gitlab # [ 0.493906] Memory: 6011316K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved)236gitlab # [ 0.495017] devtmpfs: initialized237gitlab # [ 0.496272] x86/mm: Memory block size: 128MB238gitlab # [ 0.498782] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.499699] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.500699] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.501951] PM: RTC time: 22:09:52, date: 2026-09-18242gitlab # [ 0.505976] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.508173] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.508675] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.510890] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.511620] audit: initializing netlink subsys (disabled)247gitlab # [ 0.512667] audit: type=2000 audit(1789769393.021:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.512912] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.514607] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.515607] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.516606] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.517607] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.518621] cpuidle: using governor menu254gitlab # [ 0.520832] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.521865] PCI: Using configuration type 1 for base access256gitlab # [ 0.522608] PCI: Using configuration type 1 for extended access257gitlab # [ 0.523736] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.526871] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.527609] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.528607] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.529606] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.535133] ACPI: Added _OSI(Module Device)263gitlab # [ 0.536613] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.538616] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.545625] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.552652] ACPI: Interpreter enabled267gitlab # [ 0.553670] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.555615] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.557679] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.561616] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.564953] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.577856] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.580679] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.584387] acpiphp: Slot [3] registered275gitlab # [ 0.585654] acpiphp: Slot [4] registered276gitlab # [ 0.586652] acpiphp: Slot [5] registered277gitlab # [ 0.588672] acpiphp: Slot [6] registered278gitlab # [ 0.589653] acpiphp: Slot [7] registered279gitlab # [ 0.590652] acpiphp: Slot [8] registered280gitlab # [ 0.592668] acpiphp: Slot [9] registered281gitlab # [ 0.593654] acpiphp: Slot [10] registered282gitlab # [ 0.595662] acpiphp: Slot [11] registered283gitlab # [ 0.596655] acpiphp: Slot [12] registered284gitlab # [ 0.597670] acpiphp: Slot [13] registered285gitlab # [ 0.599652] acpiphp: Slot [14] registered286gitlab # [ 0.600649] acpiphp: Slot [15] registered287gitlab # [ 0.602649] acpiphp: Slot [16] registered288gitlab # [ 0.603637] acpiphp: Slot [17] registered289gitlab # [ 0.604381] acpiphp: Slot [18] registered290gitlab # [ 0.604626] acpiphp: Slot [19] registered291gitlab # [ 0.605357] acpiphp: Slot [20] registered292gitlab # [ 0.606639] acpiphp: Slot [21] registered293gitlab # [ 0.607379] acpiphp: Slot [22] registered294gitlab # [ 0.607626] acpiphp: Slot [23] registered295gitlab # [ 0.608383] acpiphp: Slot [24] registered296gitlab # [ 0.609629] acpiphp: Slot [25] registered297gitlab # [ 0.610372] acpiphp: Slot [26] registered298gitlab # [ 0.610626] acpiphp: Slot [27] registered299gitlab # [ 0.611381] acpiphp: Slot [28] registered300gitlab # [ 0.612635] acpiphp: Slot [29] registered301gitlab # [ 0.613373] acpiphp: Slot [30] registered302gitlab # [ 0.613877] acpiphp: Slot [31] registered303gitlab # [ 0.615208] PCI host bridge to bus 0000:00304gitlab # [ 0.615622] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.616608] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.617608] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.619608] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.620607] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.621619] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.622916] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.625169] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.628705] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.633667] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.635666] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.637612] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.639611] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.641611] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.643967] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.648290] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.650775] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.651887] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.653615] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.654824] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.668672] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.669621] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.670634] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.671784] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.673376] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.678921] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.679561] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.680399] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.707627] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.713088] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.721627] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.723614] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.724630] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.726888] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.730626] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.731548] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.732413] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.735844] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.739621] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.740538] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.741666] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.743603] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.749628] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.750614] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.751630] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.753903] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.758625] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.759528] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.760630] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.762856] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.766627] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.767532] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.768413] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.769578] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.771855] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.774620] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.775649] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.777818] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.781626] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.782530] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.783413] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.786817] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.789617] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.790517] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.792406] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.797383] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.798571] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.799531] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.800538] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.802638] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.805716] iommu: Default domain type: Translated375gitlab # [ 0.806607] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.807924] ACPI: bus type USB registered377gitlab # [ 0.809212] usbcore: registered new interface driver usbfs378gitlab # [ 0.811853] usbcore: registered new interface driver hub379gitlab # [ 0.813720] usbcore: registered new device driver usb380gitlab # [ 0.820117] NetLabel: Initializing381gitlab # [ 0.821614] NetLabel: domain hash size = 128382gitlab # [ 0.823612] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.825848] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.827636] PCI: Using ACPI for IRQ routing385gitlab # [ 0.830734] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.831603] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.831603] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.838623] vgaarb: loaded389gitlab # [ 0.841106] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.842611] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.849104] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.852642] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.854437] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.857913] pnp: PnP ACPI init395gitlab # [ 0.861112] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.875167] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.878206] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.880282] NET: Registered PF_INET protocol family399gitlab # [ 0.884339] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.906161] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.907706] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.909190] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.912877] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.914287] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.915518] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.916895] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.918122] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.919548] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.920558] NET: Registered PF_XDP protocol family410gitlab # [ 0.921440] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.922501] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.923579] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.924764] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.925944] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.927363] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.928367] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.930784] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.933332] PCI: CLS 0 bytes, default 64419gitlab # [ 0.934118] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.934746] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.935256] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.935384] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns423gitlab # [ 0.959101] Initialise system trusted keyrings424gitlab # [ 0.960389] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 0.971805] Key type asymmetric registered426gitlab # [ 0.972572] Asymmetric key parser 'x509' registered427gitlab # [ 0.973519] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 0.975213] io scheduler mq-deadline registered429gitlab # [ 0.976039] io scheduler kyber registered430gitlab # [ 0.980469] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 0.981825] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 0.984317] Linux agpgart interface v0.103433gitlab # [ 0.985105] ACPI: bus type drm_connector registered434gitlab # [ 0.988423] usbcore: registered new interface driver usbserial_generic435gitlab # [ 0.989566] usbserial: USB Serial support registered for generic436gitlab # [ 0.990624] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 0.990624] Please enable it if your BIOS has the CPPC option.438gitlab # [ 0.992998] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 0.994479] drop_monitor: Initializing network drop monitor service440gitlab # [ 0.995687] NET: Registered PF_INET6 protocol family441gitlab # [ 1.000006] Segment Routing with IPv6442gitlab # [ 1.000755] In-situ OAM (IOAM) with IPv6443gitlab # [ 1.003373] IPI shorthand broadcast: enabled444gitlab # [ 1.007858] sched_clock: Marking stable (859011662, 148020350)->(1032122679, -25090667)445gitlab # [ 1.009738] registered taskstats version 1446gitlab # [ 1.010792] Loading compiled-in X.509 certificates447gitlab # [ 1.020164] Demotion targets for Node 0: null448gitlab # [ 1.021309] Key type .fscrypt registered449gitlab # [ 1.022039] Key type fscrypt-provisioning registered450gitlab # [ 1.023063] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.024038] ima: Allocated hash algorithm: sha1452gitlab # [ 1.024893] ima: No architecture policies found453gitlab # [ 1.026004] PM: Magic number: 10:323:199454gitlab # [ 1.027726] RAS: Correctable Errors collector initialized.455gitlab # [ 1.032255] clk: Disabling unused clocks456gitlab # [ 1.032994] PM: genpd: Disabling unused power domains457gitlab # [ 1.221287] Freeing initrd memory: 29392K458gitlab # [ 1.228256] Freeing unused decrypted memory: 2028K459gitlab # [ 1.234676] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.237412] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.242352] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.246268] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.315724] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.316911] Run /init as init process465gitlab # [ 1.328389] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.350128] fuse: init (API version 7.45)467gitlab # [ 1.359598] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.366484] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.370010] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.408717] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.745375] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.786472] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.788658] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.790889] systemd[1]: Running in initrd.475gitlab # [ 1.793433] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.796395] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 1.901105] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 1.957263] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 1.966950] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 1.969592] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 1.973176] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 1.975935] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 1.978161] systemd[1]: Reached target Path Units.484gitlab # [ 1.979942] systemd[1]: Reached target Slice Units.485gitlab # [ 1.981776] systemd[1]: Reached target Swaps.486gitlab # [ 1.983435] systemd[1]: Reached target Timer Units.487gitlab # [ 1.985382] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 1.987832] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 1.990198] systemd[1]: Listening on Journal Sockets.490gitlab # [ 1.992209] systemd[1]: Listening on udev Control Socket.491gitlab # [ 1.993300] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 1.994286] systemd[1]: Reached target Socket Units.493gitlab # [ 1.996274] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 1.998274] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 2.000300] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 2.010225] systemd[1]: Starting Journal Service...497gitlab # [ 2.015133] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 2.016195] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 2.020168] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 2.031712] systemd[1]: Finished Create List of Static Device Nodes.501gitlab # [ 2.038809] systemd[1]: modprobe@configfs.service: Deactivated successfully.502gitlab # [ 2.040970] systemd-journald[91]: Collecting audit messages is disabled.503gitlab # [ 2.044340] systemd[1]: Finished Load Kernel Module configfs.504gitlab # [ 2.049285] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config505gitlab # [ 2.058313] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...506gitlab # [ 2.068535] netfs: FS-Cache loaded507gitlab # [ 2.075217] 9pnet: Installing 9P2000 support508gitlab # [ 2.081190] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.509gitlab # [ 2.083839] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.510gitlab # [ 2.086383] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev511gitlab # [ 2.088349] systemd[1]: Starting Create Static Device Nodes in /dev...512gitlab # [ 2.089926] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.513gitlab # [ 2.092410] systemd[1]: Finished Load Kernel Module 9pnet_virtio.514gitlab # [ 2.094277] systemd[1]: Finished Load Kernel Modules.515gitlab # [ 2.098111] systemd[1]: Starting Apply Kernel Variables...516gitlab # [ 2.099757] systemd[1]: Finished Create Static Device Nodes in /dev.517gitlab # [ 2.101119] systemd[1]: Reached target Preparation for Local File Systems.518gitlab # [ 2.102355] systemd[1]: Reached target Local File Systems.519gitlab # [ 1.955370] systemd-modules-load[92]: Using 4 probe threads520gitlab # [ 1.956666] systemd-modules-load[92]: Inserted module 'virtio_balloon'521gitlab # [ 1.957740] systemd-modules-load[92]: Inserted module 'virtio_gpu'522gitlab # [ 2.106103] systemd[1]: Starting Rule-based Manager for Device Events and Files...523gitlab # [ 1.959661] systemd-modules-l[ 2.107572] systemd[1]: Started Journal Service.524gitlab # oad[92]: Inserted module 'dm_mod'525gitlab # [ 1.962531] systemd[1]: Finished Apply Kernel Variables.526gitlab # [ 1.966597] systemd[1]: Starting Create System Files and Directories...527gitlab # [ 1.976918] systemd-udevd[104]: Using default interface naming scheme 'v261'.528gitlab # [ 1.978096] systemd[1]: Finished Create System Files and Directories.529gitlab # [ 1.987683] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 2.006371] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 2.007479] systemd[1]: Reached target System Initialization.532gitlab # [ 2.008328] systemd[1]: Reached target Basic System.533gitlab # [ 2.234066] uhci_hcd 0000:00:01.2: UHCI Host Controller534gitlab # [ 2.236874] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1535gitlab # [ 2.238720] uhci_hcd 0000:00:01.2: detected 2 ports536gitlab # [ 2.245926] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140537gitlab # [ 2.247106] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18538gitlab # [ 2.248500] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1539gitlab # [ 2.249530] SCSI subsystem initialized540gitlab # [ 2.249794] usb usb1: Product: UHCI Host Controller541gitlab # [ 2.251265] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd542gitlab # [ 2.252181] usb usb1: SerialNumber: 0000:00:01.2543gitlab # [ 2.265266] virtio_blk virtio4: 4/0/0 default/read/poll queues544gitlab # [ 2.266173] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12545gitlab # [ 2.268099] serio: i8042 KBD port at 0x60,0x64 irq 1546gitlab # [ 2.268900] serio: i8042 AUX port at 0x60,0x64 irq 12547gitlab # [ 2.271076] hub 1-0:1.0: USB hub found548gitlab # [ 2.271714] hub 1-0:1.0: 2 ports detected549gitlab # [ 2.273452] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)550gitlab # [ 2.285905] scsi host0: ata_piix551gitlab # [ 2.287585] scsi host1: ata_piix552gitlab # [ 2.290498] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0553gitlab # [ 2.291643] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0554gitlab # [ 2.291966] virtio_blk virtio5: 4/0/0 default/read/poll queues555gitlab # [ 2.295464] virtio_blk virtio5: [vdb] 11843792 512-byte logical blocks (6.06 GB/5.65 GiB)556gitlab # [ 2.168745] (udev-worker)[138]: Network interface NamePolicy= disabled on kernel command line.557gitlab # [ 2.170143] systemd[1]: Starting Virtual Console Setup...558gitlab # [ 2.177288] (udev-worker)[121]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.559gitlab # [ 2.179302] (udev-worker)[121]: Network interface NamePolicy= disabled on kernel command line.560gitlab # [ 2.189580] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.191774] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 2.451305] ata2: found unknown device (class 0)563gitlab # [ 2.452480] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100564gitlab # [ 2.454788] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5565gitlab # [ 2.483507] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0566gitlab # [ 2.491599] usb 1-1: new full-speed USB device number 2 using uhci_hcd567gitlab # [ 2.345186] systemd[1]: Found device /dev/disk/by-label/nixos.568gitlab # [ 2.348007] systemd[1]: Reached target Initrd Root Device.569gitlab # [ 2.351589] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...570gitlab # [ 2.503184] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray571gitlab # [ 2.360074] systemd[1]: Found device /dev/disk/by-label/nix-store.572gitlab # [ 2.536768] cdrom: Uniform CD-ROM driver Revision: 3.20573gitlab # [ 2.392752] systemd-fsck[158]: nixos: clean, 12/65536 files, 13019/262144 blocks574gitlab # [ 2.399838] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.575gitlab # [ 2.659158] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.660342] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.661377] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.662001] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.662625] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.679252] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.687668] usbcore: registered new interface driver usbhid582gitlab # [ 2.688467] usbhid: USB HID core driver583gitlab # [ 2.692519] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.694195] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.883350] systemd[1]: Mounting /sysroot...586gitlab # [ 3.136652] EXT4-fs (vda): mounted filesystem 8f1917b6-b19f-4005-8af5-0990c3c16158 r/w with ordered data mode. Quota mode: none.587gitlab # [ 2.993691] systemd[1]: Mounted /sysroot.588gitlab # [ 2.995454] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 2.997518] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 2.999489] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 3.009571] systemd[1]: Mounting /sysroot/run...592gitlab # [ 3.011531] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 3.014087] systemd[1]: Mounting /sysroot/tmp/xchg...594gitlab # [ 3.017152] systemd[1]: Starting Mountpoints Configured in the Real Root...595gitlab # [ 3.026235] systemd[1]: Mounted /sysroot/nix/.rw-store.596gitlab # [ 3.028586] systemd[1]: Mounted /sysroot/run.597gitlab # [ 3.030736] systemd-sysroot-fstab-check[179]: /sysroot should be mounted in the initrd, will request daemon-reload.598gitlab # [ 3.034387] systemd[1]: Starting rw-sysroot-nix-store.service...599gitlab # [ 3.184605] 9p: Installing v9fs 9p2000 file system support600gitlab # [ 3.036759] systemd[1]: Reload requested from client PID 179 ('systemd-sysroot') (unit initrd-parse-etc.service)...601gitlab # [ 3.041257] systemd[1]: Reloading...602gitlab # [ 3.193650] erofs (device vdb): mounted with root inode @ nid 128.603gitlab # [ 3.097666] systemd[1]: Reloading finished in 61 ms.604gitlab # [ 3.118380] systemd-sysroot-fstab-check[179]: Requesting initrd-fs.target/start/replace...605gitlab # [ 3.121162] systemd[1]: Mounted /sysroot/nix/.ro-store.606gitlab # [ 3.122962] systemd[1]: Mounted /sysroot/tmp/shared.607gitlab # [ 3.124716] systemd[1]: Mounted /sysroot/tmp/xchg.608gitlab # [ 3.126789] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.129261] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.131265] systemd-sysroot-fstab-check[179]: Requesting swap.target/start/replace...611gitlab # [ 3.133789] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.135812] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.138128] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.140337] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.149662] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.152085] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.883579] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 3.907832] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 3.908626] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 3.909456] systemd[1]: Starting Find NixOS closure...621gitlab # [ 3.910230] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 3.925120] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 3.926278] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 3.933738] systemd[1]: Finished Find NixOS closure.625gitlab # [ 3.934581] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 3.935584] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 3.952378] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 3.953237] systemd[1]: Stopped target Basic System.629gitlab # [ 3.953972] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 3.954776] systemd[1]: Stopped target Path Units.631gitlab # [ 3.955497] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 3.956529] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 3.957583] systemd[1]: Stopped target Slice Units.634gitlab # [ 3.958313] systemd[1]: Stopped target Socket Units.635gitlab # [ 3.959067] systemd[1]: Stopped target System Initialization.636gitlab # [ 3.959873] systemd[1]: Stopped target Swaps.637gitlab # [ 3.960534] systemd[1]: Stopped target Timer Units.638gitlab # [ 3.961258] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 3.962090] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 3.962958] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 3.964417] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 3.965178] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 3.966754] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 3.967614] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 3.968538] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 3.969336] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 3.970386] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 3.971139] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 3.972236] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 3.973333] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 3.974355] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 3.975252] systemd[1]: Stopped target Local File Systems.653gitlab # [ 3.976046] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 3.977027] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 3.978031] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 3.978804] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 3.979839] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 3.980886] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 3.981751] systemd[1]: initrd-cleanup.service: Deactivated successfully.660gitlab # [ 3.982701] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.661gitlab # [ 3.983626] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.662gitlab # [ 3.984633] systemd[1]: Finished rw-sysroot-nix-store.service.663gitlab # [ 3.985461] systemd[1]: systemd-udevd.service: Deactivated successfully.664gitlab # [ 3.986377] systemd[1]: Stopped Rule-based Manager for Device Events and Files.665gitlab # [ 3.987369] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 3.988380] systemd[1]: Closed udev Control Socket.667gitlab # [ 3.989107] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 3.989876] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 3.991131] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 3.992073] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 3.993186] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 3.994350] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 3.995335] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 3.996227] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 3.997285] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 3.998049] systemd[1]: Reached target Switch Root.677gitlab # [ 3.998769] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.073868] initrd-nixos-activation-start[231]: booting system configuration /nix/store/16721jrf13jly83h6234wkpchnp4dgh5-nixos-system-gitlab-test679gitlab # [ 4.096660] initrd-nixos-activation-start[231]: running activation script...680gitlab # [ 4.255574] initrd-nixos-activation-start[254]: setting up /etc...681gitlab # [ 4.323738] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.324894] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.325681] systemd[1]: Starting Switch Root...684gitlab # [ 4.342370] systemd[1]: Switching root.685gitlab # [ 4.544477] systemd-journald[91]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 4.615067] NET: Registered PF_VSOCK protocol family687gitlab # [ 4.976860] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 4.981098] systemd[1]: Detected virtualization kvm.689gitlab # [ 4.981808] systemd[1]: Detected architecture x86-64.690gitlab # [ 4.982632] systemd[1]: Detected first boot.691gitlab # [ 4.983745] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.121574] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.216016] systemd[1]: Applying preset policy.694gitlab # [ 5.314707] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 5.418508] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 5.419850] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 5.421873] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 5.424098] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 5.425676] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 5.427161] systemd[1]: Created slice User and Session Slice.701gitlab # [ 5.428133] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 5.429439] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 5.430591] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 5.431350] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 5.432161] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 5.433048] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 5.433826] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 5.434721] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 5.435668] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 5.436681] systemd[1]: Reached target Path Units.711gitlab # [ 5.437508] systemd[1]: Reached target Remote File Systems.712gitlab # [ 5.438329] systemd[1]: Reached target Slice Units.713gitlab # [ 5.439093] systemd[1]: Reached target Swaps.714gitlab # [ 5.440506] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 5.442550] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 5.443932] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 5.445569] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 5.446543] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 5.448896] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 5.450345] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 5.452542] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 5.453753] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 5.454958] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 5.456418] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 5.459243] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 5.460323] systemd[1]: Listening on udev Control Socket.727gitlab # [ 5.461274] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 5.463663] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 5.465688] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 5.467806] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 5.469697] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 5.473594] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 5.475151] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 5.480111] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 5.481043] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 5.494556] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 5.496156] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 5.501033] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 5.501933] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 5.506988] systemd[1]: Starting Journal Service...741gitlab # [ 5.510078] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 5.515091] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 5.519753] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 5.520807] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 5.526081] systemd[1]: Starting Coldplug All udev Devices...746gitlab # [ 5.527744] loop: module loaded747gitlab # [ 5.529110] systemd[1]: Listening on Journal Log Access Socket.748gitlab # [ 5.530089] systemd[1]: Mounted Huge Pages File System.749gitlab # [ 5.533396] systemd[1]: Mounted POSIX Message Queue File System.750gitlab # [ 5.536152] systemd[1]: Mounted Kernel Debug File System.751gitlab # [ 5.537872] systemd-journald[325]: Collecting audit messages is enabled.752gitlab # [ 5.540504] systemd[1]: Mounted Kernel Trace File System.753gitlab # [ 5.546499] systemd[1]: Finished Create List of Static Device Nodes.754gitlab # [ 5.548057] EXT4-fs (vda): re-mounted 8f1917b6-b19f-4005-8af5-0990c3c16158.755gitlab # [ 5.550815] systemd[1]: modprobe@configfs.service: Deactivated successfully.756gitlab # [ 5.405709] systemd[1]: Queued start job for default target Multi-User System[ 5.554349] systemd[1]: Finished Load Kernel Module configfs.757gitlab # .758gitlab # [ 5.409195] systemd[1]: Unnecessary job was removed for /dev/vdb.759gitlab # [ 5.410889] systemd[1]: syste[ 5.558791] systemd[1]: Started Journal Service.760gitlab # md-journald.service: Deactivated successfully.761gitlab # [ 5.413399] systemd-modules-load[326]: Using 3 probe threads762gitlab # [ 5.414978] systemd-modules-load[326]: Inserted module 'loop'763gitlab # [ 5.416590] systemd-modules-load[326]: Inserted module 'tls'764gitlab # [ 5.418347] systemd-oomd[328]: No swap; memory pressure usage will be degraded765gitlab # [ 5.420907] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.766gitlab # [ 5.422643] systemd[1]: Mounted FUSE Control File System.767gitlab # [ 5.424105] systemd[1]: Finished Load Kernel Modules.768gitlab # [ 5.424985] systemd[1]: Finished Remount Root and Kernel File Systems.769gitlab # [ 5.431320] systemd[1]: Listening on Disk Image Download Service Socket.770gitlab # [ 5.432985] systemd[1]: Mounting Kernel Configuration File System...771gitlab # [ 5.436448] systemd[1]: Starting Firewall...772gitlab # [ 5.585208] systemd-journald[325]: Received client request to flush runtime journal.773gitlab # [ 5.476181] systemd[1]: Starting Flush Journal to Persistent Storage...774gitlab # [ 5.477460] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore775gitlab # [ 5.479193] systemd[1]: Starting Load/Save OS Random Seed...776gitlab # [ 5.480228] systemd[1]: Starting Apply Kernel Variables...777gitlab # [ 5.481286] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...778gitlab # [ 5.482348] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os779gitlab # [ 5.483550] systemd[1]: Mounted Kernel Configuration File System.780gitlab # [ 5.484421] systemd[1]: Finished Load/Save OS Random Seed.781gitlab # [ 5.485243] systemd[1]: Reached target First Boot Complete.782gitlab # [ 5.486064] systemd[1]: Finished Apply Kernel Variables.783gitlab # [ 5.486831] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.784gitlab # [ 5.487881] systemd[1]: Starting Create Static Device Nodes in /dev...785gitlab # [ 5.488739] systemd[1]: Finished Flush Journal to Persistent Storage.786gitlab # [ 5.490134] systemd[1]: Finished Create Static Device Nodes in /dev.787gitlab # [ 5.491052] systemd[1]: Reached target Preparation for Local File Systems.788gitlab # [ 5.492242] systemd[1]: Starting Rule-based Manager for Device Events and Files...789gitlab # [ 5.507271] systemd-udevd[368]: Using default interface naming scheme 'v261'.790gitlab # [ 5.537179] systemd[1]: Started Rule-based Manager for Device Events and Files.791gitlab # [ 5.554585] systemd[1]: Finished Coldplug All udev Devices.792gitlab # [ 5.601949] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 5.620901] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 5.625749] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.795gitlab # [ 5.634352] (udev-worker)[429]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.796gitlab # [ 5.636640] (udev-worker)[429]: Network interface NamePolicy= disabled on kernel command line.797gitlab # [ 5.644429] (udev-worker)[443]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 5.651125] systemd[1]: Condition check resulted in Virtio network device being skipped.799gitlab # [ 5.652235] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore800gitlab # [ 5.653628] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67801gitlab # [ 5.655625] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore802gitlab # [ 5.657120] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os803gitlab # [ 5.658329] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os804gitlab # [ 5.680542] systemd[1]: Finished Firewall.805gitlab # [ 5.836917] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3806gitlab # [ 5.838467] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console807gitlab # [ 5.842526] ACPI: button: Power Button [PWRF]808gitlab # [ 5.855656] Console: switching to colour dummy device 80x25809gitlab # [ 5.856056] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0810gitlab # [ 5.856817] [drm] Found bochs VGA, ID 0xb0c5.811gitlab # [ 5.856950] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD812gitlab # [ 5.858095] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.813gitlab # [ 5.862092] mousedev: PS/2 mouse device common for all mice814gitlab # [ 5.863048] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic815gitlab # [ 5.863941] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0816gitlab # [ 5.876163] Console: switching to colour frame buffer device 160x50817gitlab # [ 5.877372] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device818gitlab # [ 5.905361] parport_pc 00:03: reported by Plug and Play ACPI819gitlab # [ 5.906436] rtc_cmos 00:05: RTC can wake from S4820gitlab # [ 5.906453] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]821gitlab # [ 5.909159] rtc_cmos 00:05: registered as rtc0822gitlab # [ 5.909482] Floppy drive(s): fd0 is 2.88M AMI BIOS823gitlab # [ 5.909790] rtc_cmos 00:05: setting system clock to 2026-09-18T22:09:58 UTC (1789769398)824gitlab # [ 5.913101] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs825gitlab # [ 5.916813] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4826gitlab # [ 5.918046] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6827gitlab # [ 5.919472] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5828gitlab # [ 5.921862] FDC 0 is a S82078B829gitlab # [ 5.819938] systemd[1]: Starting Virtual Console Setup...830gitlab # [ 6.001036] erofs (device vdb): failed to read inode meta block (nid: 128519161): -4831gitlab # [ 5.854859] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.832gitlab # [ 5.855952] systemd[1]: Stopped Virtual Console Setup.833gitlab # [ 5.864263] systemd[1]: Starting Virtual Console Setup...834gitlab # [ 6.029661] ppdev: user-space parallel port driver835gitlab # [ 6.051343] kvm_amd: TSC scaling supported836gitlab # [ 6.052134] kvm_amd: Nested Virtualization enabled837gitlab # [ 6.052837] kvm_amd: Nested Paging enabled838gitlab # [ 6.053548] kvm_amd: LBR virtualization supported839gitlab # [ 6.054302] kvm_amd: Virtual VMLOAD VMSAVE supported840gitlab # [ 6.055074] kvm_amd: Virtual GIF supported841gitlab # [ 6.055678] kvm_amd: Virtual NMI enabled842gitlab # [ 5.932282] systemd-vconsole-setup[498]: Configuration of first virtual console was skipped, ignoring remaining ones.843gitlab # [ 5.935063] systemd[1]: Finished Virtual Console Setup.844gitlab # [ 6.089051] EDAC MC: Ver: 3.0.0845gitlab # [ 6.272918] systemd[1]: Mounting /run/wrappers...846gitlab # [ 6.294650] systemd[1]: Mounted /run/wrappers.847gitlab # [ 6.295946] systemd[1]: Reached target Local File Systems.848gitlab # [ 6.297404] systemd[1]: Listening on Boot Loader Control Service Socket.849gitlab # [ 6.299475] systemd[1]: Starting register-nix-paths.service...850gitlab # [ 6.301268] systemd[1]: Starting Create SUID/SGID Wrappers...851gitlab # [ 6.302416] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.852gitlab # [ 6.305329] systemd[1]: Starting Save Transient machine-id to Disk...853gitlab # [ 6.316561] systemd[1]: Starting Create System Files and Directories...854gitlab # [ 6.344199] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.855gitlab # [ 6.348366] systemd[1]: Finished Save Transient machine-id to Disk.856gitlab # [ 6.364873] systemd[1]: Finished Create System Files and Directories.857gitlab # [ 6.372146] systemd[1]: Starting Rebuild Journal Catalog...858gitlab # [ 6.374526] systemd[1]: Starting Record System Boot/Shutdown in UTMP...859gitlab # [ 6.391437] systemd[1]: Finished Record System Boot/Shutdown in UTMP.860gitlab # [ 6.409867] systemd[1]: Finished Rebuild Journal Catalog.861gitlab # [ 6.411339] systemd[1]: Starting Update is Completed...862gitlab # [ 6.428416] systemd[1]: Finished Update is Completed.863gitlab # [ 6.449057] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.864gitlab # [ 6.450327] systemd[1]: Finished Create SUID/SGID Wrappers.865gitlab # [ 6.625763] systemd[1]: Finished register-nix-paths.service.866gitlab # [ 6.627584] systemd[1]: Reached target System Initialization.867gitlab # [ 6.629278] systemd[1]: Started Discard unused filesystem blocks once a week.868gitlab # [ 6.631273] systemd[1]: Started logrotate.timer.869gitlab # [ 6.632717] systemd[1]: Started Daily Cleanup of Temporary Directories.870gitlab # [ 6.634584] systemd[1]: Reached target Timer Units.871gitlab # [ 6.636177] systemd[1]: Listening on D-Bus System Message Bus Socket.872gitlab # [ 6.638080] systemd[1]: Listening on Nix Daemon Socket.873gitlab # [ 6.644652] systemd[1]: Starting nixbot CI service socket...874gitlab # [ 6.646286] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.875gitlab # [ 6.651439] systemd[1]: Starting D-Bus System Message Bus...876gitlab # [ 6.653212] systemd[1]: Listening on nixbot CI service socket.877gitlab # [ 6.654895] systemd[1]: Reached target Socket Units.878gitlab # [ 6.680988] dbus-broker-launch[580]: Looking up NSS user entry for 'systemd-timesync'...879gitlab # [ 6.683853] dbus-broker-launch[580]: NSS returned no entry for 'systemd-timesync'880gitlab # [ 6.686142] dbus-broker-launch[580]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"881gitlab # [ 6.690398] systemd[1]: Started D-Bus System Message Bus.882gitlab # [ 6.703131] dbus-broker-launch[580]: Ready883gitlab # [ 6.705672] systemd[1]: Reached target Basic System.884gitlab # [ 6.707621] systemd[1]: Started backdoor.service.885gitlab # [ 6.710687] systemd[1]: Starting gitlab-config.service...886gitlab # [ 6.723254] systemd[1]: Starting Import lastlog data into lastlog2 database...887gitlab # [ 6.727315] systemd[1]: Starting Logrotate configuration check...888gitlab # [ 6.732494] systemd[1]: Starting Name Service Cache Daemon (nsncd)...889gitlab # [ 6.738524] systemd[1]: Starting Post-Boot Actions...890gitlab # connecting to host...891gitlab # [ 6.746804] logrotate[585]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.892gitlab # [ 6.751078] logrotate[585]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf893gitlab # [ 6.754928] logrotate[585]: note: 'monthly' overrides previously specified 'weekly'894gitlab # [ 6.755468] systemd[1]: Started Reset console on configuration changes.895gitlab # [ 6.756379] logrotate[585]: Reading state from file: /var/lib/logrotate.status896gitlab # [ 6.757191] logrotate[585]: state file /var/lib/logrotate.status does not exist897gitlab # [ 6.760265] logrotate[585]: Allocating hash table for state file, size 64 entries898gitlab # [ 6.761916] logrotate[585]: Handling 4 logs899gitlab # [ 6.762798] logrotate[585]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed900gitlab # [ 6.763607] logrotate[585]: considering log /var/log/btmp901gitlab # [ 6.764458] logrotate[585]: Creating new state902gitlab # [ 6.766095] logrotate[585]: Now: 2026-09-18 22:09903gitlab # [ 6.766519] logrotate[585]: Last rotated at 2026-09-18 22:00904gitlab: Guest shell says: b'Spawning backdoor root shell...\n'905gitlab # [ 6.766996] logrotate[585]: log does not need rotating (log has already been rotated)906gitlab # [ 6.767739] logrotate[585]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed907gitlab # [ 6.768499] logrotate[585]: considering log /var/log/wtmp908gitlab # [ 6.769680] logrotate[585]: Creating new state909gitlab: connected to guest root shell910gitlab # [ 6.770504] logrotate[585]: Now: 2026-09-18 22:09911gitlab: (connecting took 7.41 seconds)912gitlab # [ 6.771089] logrotate[585]: Last rotated at 2026-09-18 22:00913gitlab: (finished: waiting for the VM to finish booting, in 7.41 seconds)914gitlab # [ 6.771536] logrotate[585]: log does not need rotating (log has already been rotated)915gitlab # [ 6.772143] logrotate[585]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed916gitlab # [ 6.772735] logrotate[585]: switching euid from 0 to 165 and egid from 0 to 165 (pid 585)917gitlab # [ 6.773478] logrotate[585]: considering log /var/gitlab/state/log/*.log918gitlab # [ 6.774098] logrotate[585]: log /var/gitlab/state/log/*.log does not exist -- skipping919gitlab # [ 6.774654] logrotate[585]: Creating new state920gitlab # [ 6.775090] logrotate[585]: switching euid from 165 to 0 and egid from 165 to 0 (pid 585)921gitlab # [ 6.775540] logrotate[585]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed922gitlab # [ 6.775966] logrotate[585]: switching euid from 0 to 60 and egid from 0 to 60 (pid 585)923gitlab # [ 6.776569] logrotate[585]: considering log /var/log/nginx/*.log924gitlab # [ 6.777156] logrotate[585]: log /var/log/nginx/*.log does not exist -- skipping925gitlab # [ 6.777603] logrotate[585]: Creating new state926gitlab # [ 6.778044] logrotate[585]: not running postrotate script, since no logs were rotated927gitlab # [ 6.778486] logrotate[585]: switching euid from 60 to 0 and egid from 60 to 0 (pid 585)928gitlab # [ 6.779036] systemd[1]: Starting resolvconf update...929gitlab # [ 6.779774] systemd[1]: Finished Logrotate configuration check.930gitlab # [ 6.780535] systemd[1]: Finished Post-Boot Actions.931gitlab # [ 6.787894] systemd[1]: Finished Import lastlog data into lastlog2 database.932gitlab # [ 6.805713] nsncd[586]: Sep 18 22:09:59.543 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"933gitlab # [ 6.806129] systemd[1]: Started Name Service Cache Daemon (nsncd).934gitlab # [ 6.809089] systemd[1]: Reached target Host and Network Name Lookups.935gitlab # [ 6.809603] systemd[1]: Reached target User and Group Name Lookups.936gitlab # [ 6.815057] systemd[1]: Starting User Login Management...937gitlab # [ 6.836143] systemd[1]: Stopped target Host and Network Name Lookups.938gitlab # [ 6.837280] systemd[1]: Stopping Host and Network Name Lookups...939gitlab # [ 6.837782] systemd[1]: Stopped target User and Group Name Lookups.940gitlab # [ 6.838419] systemd[1]: Stopping User and Group Name Lookups...941gitlab # [ 6.838915] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...942gitlab # [ 6.841966] systemd[1]: nscd.service: Deactivated successfully.943gitlab # [ 6.842511] systemd[1]: Stopped Name Service Cache Daemon (nsncd).944gitlab # [ 6.843167] systemd[1]: nscd.service: Consumed 25ms CPU time over 109ms wall clock time, 7.1M memory peak, 5.5M read from disk.945gitlab # [ 6.845906] systemd[1]: Starting Name Service Cache Daemon (nsncd)...946gitlab # [ 6.855558] systemd-logind[628]: New seat seat0.947gitlab # [ 6.861344] systemd-logind[628]: Watching system buttons on /dev/input/event2 (Power Button)948gitlab # [ 6.862648] systemd-logind[628]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)949gitlab # [ 6.864290] systemd-logind[628]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)950gitlab # [ 6.865923] nsncd[676]: Sep 18 22:09:59.601 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"951gitlab # [ 6.868314] systemd[1]: Started User Login Management.952gitlab # [ 6.869162] systemd[1]: Finished resolvconf update.953gitlab # [ 6.869938] systemd[1]: Started Name Service Cache Daemon (nsncd).954gitlab # [ 6.870916] systemd[1]: Reached target Preparation for Network.955gitlab # [ 6.871842] systemd[1]: Reached target Host and Network Name Lookups.956gitlab # [ 6.872839] systemd[1]: Reached target User and Group Name Lookups.957gitlab # [ 6.873862] systemd[1]: Starting DHCP Client...958gitlab # [ 6.874651] systemd[1]: Starting linger-users.service...959gitlab # [ 6.875430] systemd[1]: Starting Address configuration of eth1...960gitlab # [ 6.878343] systemd[1]: Starting Extra networking commands....961gitlab # [ 6.890329] systemd[1]: linger-users.service: Deactivated successfully.962gitlab # [ 6.892655] systemd[1]: Finished linger-users.service.963gitlab # [ 6.896979] network-addresses-eth1-start[701]: adding address 192.168.1.1/24... done964gitlab # [ 6.901156] network-addresses-eth1-start[701]: adding address 2001:db8:1::1/64... done965gitlab # [ 6.906117] systemd[1]: Finished Address configuration of eth1.966gitlab # [ 6.932811] systemd[1]: Finished Extra networking commands..967gitlab # [ 6.933774] systemd[1]: Reached target Network.968gitlab # [ 6.935183] systemd[1]: Starting gitlab-workhorse.service...969gitlab # [ 6.937826] systemd[1]: Starting Nginx Web Server...970gitlab # [ 6.941321] systemd[1]: Starting PostgreSQL Server...971gitlab # [ 6.944954] systemd[1]: Starting Redis Server - redis-gitlab...972gitlab # [ 6.947461] systemd[1]: Starting Permit User Sessions...973gitlab # [ 6.949561] dhcpcd[719]: dhcpcd-10.3.2 starting974gitlab # [ 6.951280] dhcpcd[773]: dev: loaded udev975gitlab # [ 7.105153] 8021q: 802.1Q VLAN Support v1.8976gitlab # [ 7.105763] 8021q: adding VLAN 0 to HW filter on device eth1977gitlab # [ 6.962057] systemd[1]: Finished Permit User Sessions.978gitlab # [ 6.969845] systemd[1]: Started Getty on tty1.979gitlab # [ 6.972794] systemd[1]: Reached target Login Prompts.980gitlab # [ 6.976265] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.981gitlab # [ 7.198986] cfg80211: Loading compiled-in X.509 certificates for regulatory database982gitlab # [ 7.210292] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'983gitlab # [ 7.213138] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'984gitlab # [ 7.067272] postgresql-pre-st[ 7.215315] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2985gitlab # art[794]: The files belonging to[ 7.216593] cfg80211: failed to load regulatory.db986gitlab # this database system will be owned by user "postgres".987gitlab # [ 7.067448] postgresql-pre-start[794]: This user must also own the server process.988gitlab # [ 7.069378] redis-server[791]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.989gitlab # [ 7.071528] redis-server[791]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo990gitlab # [ 7.072255] redis-server[791]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=791, just started991gitlab # [ 7.073236] systemd[1]: Started Redis Server - redis-gitlab.992gitlab # [ 7.073744] redis-server[791]: Configuration loaded993gitlab # [ 7.074197] redis[791]: monotonic clock: POSIX clock_gettime994gitlab # [ 7.074707] redis[791]: Running mode=standalone, port=0.995gitlab # [ 7.075223] redis[791]: Server initialized996gitlab # [ 7.075614] redis[791]: Ready to accept connections unix997gitlab # [ 7.076137] redis[791]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.998gitlab # [ 7.083129] postgresql-pre-start[794]: The database cluster will be initialized with locale "en_US.UTF-8".999gitlab # [ 7.083252] postgresql-pre-start[794]: The default database encoding has accordingly been set to "UTF8".1000gitlab # [ 7.083786] postgresql-pre-start[794]: The default text search configuration will be set to "english".1001gitlab # [ 7.084538] postgresql-pre-start[794]: Data page checksums are enabled.1002gitlab # [ 7.085082] postgresql-pre-start[794]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1003gitlab # [ 7.085592] postgresql-pre-start[794]: creating subdirectories ... ok1004gitlab # [ 7.086103] postgresql-pre-start[794]: selecting dynamic shared memory implementation ... posix1005gitlab # [ 7.090557] nginx-pre-start[798]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok1006gitlab # [ 7.090960] nginx-pre-start[798]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful1007gitlab # [ 7.098205] systemd[1]: Started Nginx Web Server.1008gitlab # [ 7.254844] 8021q: adding VLAN 0 to HW filter on device eth01009gitlab # [ 7.109126] dhcpcd[773]: eth0: waiting for carrier1010gitlab # [ 7.109499] dhcpcd[773]: eth0: carrier acquired1011gitlab # [ 7.112776] dhcpcd[773]: DUID 00:01:00:01:32:40:73:37:52:54:00:12:34:561012gitlab # [ 7.113117] dhcpcd[773]: eth0: IAID 00:12:34:561013gitlab # [ 7.113583] dhcpcd[773]: eth0: adding address fe80::5054:ff:fe12:34561014gitlab # [ 7.121461] postgresql-pre-start[794]: selecting default "max_connections" ... 1001015gitlab # [ 7.162929] postgresql-pre-start[794]: selecting default "shared_buffers" ... 128MB1016gitlab # [ 7.406548] systemd[1]: Started gitlab-workhorse.service.1017gitlab # [ 7.523383] postgresql-pre-start[794]: selecting default time zone ... UTC1018gitlab # [ 7.524615] postgresql-pre-start[794]: creating configuration files ... ok1019gitlab # [ 7.562132] gitlab-workhorse[828]: time="2026-09-18T22:10:00Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41020gitlab # [ 7.562504] gitlab-workhorse[828]: time="2026-09-18T22:10:00Z" level=info msg="Using redis/go-redis"1021gitlab # [ 7.563050] gitlab-workhorse[828]: time="2026-09-18T22:10:00Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1022gitlab # [ 7.695711] postgresql-pre-start[794]: running bootstrap script ... ok1023gitlab # [ 8.136528] postgresql-pre-start[794]: performing post-bootstrap initialization ... ok1024gitlab # [ 8.291592] dhcpcd[773]: eth0: soliciting a DHCP lease1025gitlab # [ 8.449551] NET: Registered PF_PACKET protocol family1026gitlab # [ 8.313898] dhcpcd[773]: eth0: offered 10.0.2.15 from 10.0.2.21027gitlab # [ 8.318437] dhcpcd[773]: eth0: probing address 10.0.2.15/241028gitlab # [ 8.345782] postgresql-pre-start[794]: syncing data to disk ... ok1029gitlab # [ 8.346504] postgresql-pre-start[794]: initdb: warning: enabling "trust" authentication for local connections1030gitlab # [ 8.347093] postgresql-pre-start[794]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1031gitlab # [ 8.347758] postgresql-pre-start[794]: Success. You can now start the database server using:1032gitlab # [ 8.348513] postgresql-pre-start[794]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1033gitlab # [ 8.451610] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1034gitlab # [ 8.452390] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 54321035gitlab # [ 8.453493] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 54321036gitlab # [ 8.455495] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1037gitlab # [ 8.467190] postgres[863]: [863] LOG: database system was shut down at 2026-09-18 22:10:00 GMT1038gitlab # [ 8.474210] postgres[854]: [854] LOG: database system is ready to accept connections1039gitlab # [ 8.478118] systemd[1]: Started PostgreSQL Server.1040gitlab # [ 8.482264] systemd[1]: Starting PostgreSQL Setup Scripts...1041gitlab # [ 8.639746] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[873]: mkdir -p /var/gitlab/state/home/.ssh: OK1042gitlab # [ 8.642408] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[873]: chmod 700 /var/gitlab/state/home/.ssh: OK1043gitlab # [ 8.690308] postgresql-setup-start[879]: CREATE DATABASE1044gitlab # [ 8.694706] systemd[1]: Finished gitlab-config.service.1045gitlab # [ 8.695790] systemd[1]: Started gitaly.service.1046gitlab # [ 8.715373] postgresql-setup-start[897]: CREATE ROLE1047gitlab # [ 8.726116] postgresql-setup-start[900]: ALTER DATABASE1048gitlab # [ 8.742611] postgresql-setup-start[905]: CREATE ROLE1049gitlab # [ 8.743709] systemd[1]: Finished PostgreSQL Setup Scripts.1050gitlab # [ 8.744623] systemd[1]: Reached target PostgreSQL.1051gitlab # [ 8.746395] systemd[1]: Starting gitlab-db-config.service...1052gitlab # [ 8.747852] systemd[1]: Starting gitlab-postgresql.service...1053gitlab # [ 8.855761] gitlab-postgresql-start[915]: CREATE DATABASE1054gitlab # [ 8.891411] gitlab-postgresql-start[929]: CREATE EXTENSION1055gitlab # [ 8.965544] gitlab-postgresql-start[936]: CREATE EXTENSION1056gitlab # [ 8.969172] systemd[1]: Finished gitlab-postgresql.service.1057gitlab # [ 9.154414] gitaly[895]: time="2026-09-18T22:10:01.891Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=8951058gitlab # [ 9.155233] gitaly[895]: time="2026-09-18T22:10:01.892Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=8951059gitlab # [ 9.156139] gitaly[895]: time="2026-09-18T22:10:01.892Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=8951060gitlab # [ 9.157130] gitaly[895]: time="2026-09-18T22:10:01.892Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=8951061gitlab # [ 9.157950] gitaly[895]: time="2026-09-18T22:10:01.892Z" level=info msg="Starting Gitaly" pid=895 version=1062gitlab # [ 9.354881] gitaly[895]: time="2026-09-18T22:10:02.092Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=189 pid=8951063gitlab # [ 9.355147] gitaly[895]: time="2026-09-18T22:10:02.092Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=8951064gitlab # [ 9.355573] gitaly[895]: time="2026-09-18T22:10:02.093Z" level=info msg="finished initializing command factory" duration_ms=0 pid=8951065gitlab # [ 9.356085] gitaly[895]: time="2026-09-18T22:10:02.093Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-895/git-exec-254260051.d/git pid=8951066gitlab # [ 9.359544] gitaly[895]: time="2026-09-18T22:10:02.097Z" level=info msg="finished detecting git version" duration_ms=4 pid=8951067gitlab # [ 9.360210] gitaly[895]: time="2026-09-18T22:10:02.098Z" level=info msg="using Git version" pid=895 version=2.53.g1068gitlab # [ 9.360662] gitaly[895]: time="2026-09-18T22:10:02.098Z" level=info msg="clearing disk cache object folder" pid=895 storage=default1069gitlab # [ 9.361131] gitaly[895]: time="2026-09-18T22:10:02.098Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache484638423 pid=895 storage=default1070gitlab # [ 9.361678] gitaly[895]: time="2026-09-18T22:10:02.098Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache484638423 pid=895 storage=default1071gitlab # [ 9.362170] gitaly[895]: time="2026-09-18T22:10:02.098Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=8951072gitlab # [ 9.362628] gitaly[895]: time="2026-09-18T22:10:02.098Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=8951073gitlab # [ 9.363148] gitaly[895]: time="2026-09-18T22:10:02.099Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache484638423 clear_duration_ms=0 pid=895 storage=default1074gitlab # [ 9.393969] gitaly[895]: time="2026-09-18T22:10:02.131Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=8951075gitlab # [ 9.394300] gitaly[895]: time="2026-09-18T22:10:02.131Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=895 schema=unix1076gitlab # [ 9.394910] gitaly[895]: time="2026-09-18T22:10:02.131Z" level=info msg="counting repositories" pid=8951077gitlab # [ 9.395580] gitaly[895]: time="2026-09-18T22:10:02.132Z" level=info msg="starting to count repositories" pid=895 storage_path=/var/gitlab/state/repositories1078gitlab # [ 9.396058] gitaly[895]: time="2026-09-18T22:10:02.132Z" level=info msg="finished tempdir cleaner walk" pid=895 storage=default time_ms=01079gitlab # [ 9.396507] gitaly[895]: time="2026-09-18T22:10:02.132Z" level=info msg="listening at address" address=/run/gitaly/gitaly-895/sock.d/intern pid=895 schema=unix1080gitlab # [ 9.403965] dhcpcd[773]: eth0: soliciting an IPv6 router1081gitlab # [ 9.404468] gitaly[895]: time="2026-09-18T22:10:02.141Z" level=info msg="completed counting repositories" counting_duration_ms=9 pid=895 storage_path=/var/gitlab/state/repositories1082gitlab # [ 9.404903] gitaly[895]: time="2026-09-18T22:10:02.141Z" level=info msg="completed counting all repositories" counting_duration=9 pid=8951083gitlab # [ 9.405513] gitaly[895]: time="2026-09-18T22:10:02.141Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=8951084gitlab # [ 9.406036] gitaly[895]: time="2026-09-18T22:10:02.141Z" level=info msg="Started Gitaly" duration_ms=249 pid=8951085gitlab # [ 9.406572] gitaly[895]: time="2026-09-18T22:10:02.142Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=8951086gitlab # [ 9.407077] gitaly[895]: time="2026-09-18T22:10:02.142Z" level=info msg="maintenance: daily scheduled" pid=895 scheduled="2026-09-19 12:00:00 +0000 UTC"1087gitlab # [ 9.407547] dhcpcd[773]: eth0: Router Advertisement from fe80::21088gitlab # [ 9.408113] dhcpcd[773]: eth0: adding address fec0::5054:ff:fe12:3456/641089gitlab # [ 9.408577] dhcpcd[773]: eth0: adding route to fec0::/641090gitlab # [ 9.409079] dhcpcd[773]: eth0: adding default route via fe80::21091gitlab # [ 9.866620] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[910]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1092gitlab # [ 9.867447] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[910]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1093gitlab # [ 9.889071] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[910]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1094gitlab # [ 11.382461] gitaly[895]: time="2026-09-18T22:10:04.119Z" level=info msg="License database preloaded" duration_ms=2227 pid=8951095gitlab # [ 13.953642] dhcpcd[773]: eth0: leased 10.0.2.15 for 86400 seconds1096gitlab # [ 13.953862] dhcpcd[773]: eth0: adding route to 10.0.2.0/241097gitlab # [ 13.955031] dhcpcd[773]: eth0: adding default route via 10.0.2.21098gitlab # [ 14.048546] systemd[1]: Started DHCP Client.1099gitlab # [ 14.051426] systemd[1]: Reached target Network is Online.1100gitlab # [ 38.961354] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[910]: W, [2026-09-18T22:10:31.698583 #910] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1101gitlab # [ 41.161976] postgres[1301]: [1301] ERROR: relation "feature_gates" does not exist at character 5231102gitlab # [ 41.164322] postgres[1301]: [1301] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1103gitlab # [ 41.165436] postgres[1301]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1104gitlab # [ 41.167294] postgres[1301]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1105gitlab # [ 41.168881] postgres[1301]: attidentity AS identity,1106gitlab # [ 41.171682] postgres[1301]: attgenerated as attgenerated1107gitlab # [ 41.173508] postgres[1301]: FROM pg_attribute a1108gitlab # [ 41.175467] postgres[1301]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1109gitlab # [ 41.177417] postgres[1301]: LEFT JOIN pg_type t ON a.atttypid = t.oid1110gitlab # [ 41.179295] postgres[1301]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1111gitlab # [ 41.180910] postgres[1301]: WHERE a.attrelid = '"feature_gates"'::regclass1112gitlab # [ 41.183253] postgres[1301]: AND a.attnum > 0 AND NOT a.attisdropped1113gitlab # [ 41.185135] postgres[1301]: ORDER BY a.attnum1114gitlab # [ 41.186390] postgres[1301]:1115gitlab # [ 44.533880] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1343]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1116gitlab # [ 44.534621] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1343]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1117gitlab # [ 47.617133] gitlab-workhorse[828]: time="2026-09-18T22:10:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1118gitlab # [ 57.627186] gitlab-workhorse[828]: time="2026-09-18T22:10:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1119gitlab # [ 59.059945] postgres[1409]: [1409] ERROR: relation "table_without_models" does not exist1120gitlab # [ 59.060761] postgres[1409]: [1409] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1121gitlab # [ 67.636560] gitlab-workhorse[828]: time="2026-09-18T22:11:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 70.373894] systemd[1]: Finished gitlab-db-config.service.1123gitlab # [ 70.379721] systemd[1]: Started gitlab-sidekiq.service.1124gitlab # [ 70.383728] systemd[1]: Starting gitlab.service...1125gitlab # [ 70.727661] bundle[1576]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1126gitlab # [ 70.728138] bundle[1576]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1127gitlab # [ 70.732575] bundle[1576]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1128gitlab # [ 73.444752] sidekiq-cluster[1575]: {"severity":"INFO","time":"2026-09-18T22:11:06.164Z","message":"Starting cluster with 1 processes"}1129gitlab # [ 73.537478] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.249Z","pid":1576,"message":"* Enabling systemd notification integration"}1130gitlab # [ 73.538251] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.274Z","pid":1576,"message":"Puma starting in cluster mode..."}1131gitlab # [ 73.538951] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.275Z","pid":1576,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1132gitlab # [ 73.539739] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.275Z","pid":1576,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1133gitlab # [ 73.540762] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.275Z","pid":1576,"message":"* Min threads: 0"}1134gitlab # [ 73.541679] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.275Z","pid":1576,"message":"* Max threads: 4"}1135gitlab # [ 73.542563] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.275Z","pid":1576,"message":"* Environment: production"}1136gitlab # [ 73.543306] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.275Z","pid":1576,"message":"* Master PID: 1576"}1137gitlab # [ 73.544104] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.275Z","pid":1576,"message":"* Workers: 2"}1138gitlab # [ 73.544812] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.275Z","pid":1576,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1139gitlab # [ 73.545445] bundle[1576]: {"timestamp":"2026-09-18T22:11:06.275Z","pid":1576,"message":"* Preloading application"}1140gitlab # [ 77.646415] gitlab-workhorse[828]: time="2026-09-18T22:11:10Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1141gitlab # [ 87.647859] gitlab-workhorse[828]: time="2026-09-18T22:11:20Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1142gitlab # [ 97.648773] gitlab-workhorse[828]: time="2026-09-18T22:11:30Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1143gitlab # [ 97.753043] bundle[1576]: W, [2026-09-18T22:11:30.490800 #1576] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1144gitlab # [ 103.026135] sidekiq-cluster[1600]: 2026-09-18T22:11:35.763Z pid=1600 tid=wko INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1600, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430968832}1145gitlab # [ 103.215314] sidekiq-cluster[1600]: W, [2026-09-18T22:11:35.953192 #1600] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1146gitlab # [ 103.257845] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:35.995Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1147gitlab # [ 105.209709] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:37.946Z","message":"GitLab reliable fetch activated!"}1148gitlab # [ 105.210407] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:37.947Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1149gitlab # [ 105.240240] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:37.978Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1150gitlab # [ 105.244362] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:37.982Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1151gitlab # [ 105.248892] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:37.987Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1152gitlab # [ 105.253279] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:37.991Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1153gitlab # [ 105.257857] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:37.995Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1154gitlab # [ 105.262273] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.000Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1155gitlab # [ 105.267487] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.005Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1156gitlab # [ 105.272111] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.010Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1157gitlab # [ 105.276917] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.014Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1158gitlab # [ 105.281820] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.019Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1159gitlab # [ 105.286729] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.024Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1160gitlab # [ 105.292956] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.031Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1161gitlab # [ 105.297855] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.035Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1162gitlab # [ 105.302863] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.040Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1163gitlab # [ 105.308446] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.046Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1164gitlab # [ 105.314082] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.052Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1165gitlab # [ 105.320610] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.058Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1166gitlab # [ 105.327289] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.065Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1167gitlab # [ 105.334510] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.072Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1168gitlab # [ 105.341083] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.079Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1169gitlab # [ 105.347729] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.085Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1170gitlab # [ 105.352669] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.090Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1171gitlab # [ 105.357558] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.095Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1172gitlab # [ 105.362099] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.100Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1173gitlab # [ 105.367805] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.105Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1174gitlab # [ 105.373582] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.111Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1175gitlab # [ 105.379242] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.117Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1176gitlab # [ 105.383852] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.121Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1177gitlab # [ 105.388087] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.126Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1178gitlab # [ 105.393654] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.131Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1179gitlab # [ 105.398374] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.136Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1180gitlab # [ 105.403339] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.141Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1181gitlab # [ 105.408103] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.146Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1182gitlab # [ 105.412874] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.150Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1183gitlab # [ 105.417697] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.155Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1184gitlab # [ 105.420727] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.158Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1185gitlab # [ 105.423643] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.161Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1186gitlab # [ 105.426932] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.165Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1187gitlab # [ 105.429807] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.167Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1188gitlab # [ 105.432657] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.170Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1189gitlab # [ 105.435792] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.173Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1190gitlab # [ 105.438769] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.176Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1191gitlab # [ 105.441783] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.179Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1192gitlab # [ 105.445097] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.183Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1193gitlab # [ 105.448574] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.186Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1194gitlab # [ 105.451566] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.189Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1195gitlab # [ 105.454442] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.192Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1196gitlab # [ 105.457512] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.195Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1197gitlab # [ 105.461211] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.199Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1198gitlab # [ 105.464352] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.202Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1199gitlab # [ 105.467750] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.205Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1200gitlab # [ 105.471134] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.209Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1201gitlab # [ 105.480397] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.218Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1202gitlab # [ 105.484109] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.222Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1203gitlab # [ 105.487164] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.225Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1204gitlab # [ 105.490345] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.228Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1205gitlab # [ 105.493368] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.231Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1206gitlab # [ 105.496777] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.234Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1207gitlab # [ 105.499726] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.237Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1208gitlab # [ 105.502448] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.240Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1209gitlab # [ 105.505528] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.243Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1210gitlab # [ 105.508318] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.246Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1211gitlab # [ 105.511857] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.249Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1212gitlab # [ 105.546076] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.283Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1213gitlab # [ 105.549382] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.287Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1214gitlab # [ 105.552880] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.290Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1215gitlab # [ 105.556377] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.294Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1216gitlab # [ 105.560874] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.298Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1217gitlab # [ 105.564387] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.302Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1218gitlab # [ 105.568934] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.307Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1219gitlab # [ 105.571427] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.309Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1220gitlab # [ 105.574603] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.312Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1221gitlab # [ 105.577148] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.315Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1222gitlab # [ 105.579675] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.317Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1223gitlab # [ 105.582674] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.320Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1224gitlab # [ 105.585419] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.323Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1225gitlab # [ 105.590490] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.328Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1226gitlab # [ 105.593422] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.331Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1227gitlab # [ 105.596750] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.334Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1228gitlab # [ 105.599449] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.337Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1229gitlab # [ 105.602139] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.340Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1230gitlab # [ 105.605275] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.343Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1231gitlab # [ 105.608784] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.346Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1232gitlab # [ 105.611454] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.349Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1233gitlab # [ 105.614257] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.352Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1234gitlab # [ 105.616717] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.354Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1235gitlab # [ 105.620090] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.358Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1236gitlab # [ 105.622549] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.360Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1237gitlab # [ 105.625102] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.363Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1238gitlab # [ 105.627617] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.365Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1239gitlab # [ 105.630054] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.368Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1240gitlab # [ 105.632786] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.370Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1241gitlab # [ 105.635293] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.373Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1242gitlab # [ 105.637903] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.376Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1243gitlab # [ 105.640140] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.378Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1244gitlab # [ 105.642730] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.380Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1245gitlab # [ 105.644480] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.382Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1246gitlab # [ 105.646930] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.385Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1247gitlab # [ 105.655727] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.393Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1248gitlab # [ 105.657696] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.395Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1249gitlab # [ 105.659614] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.397Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1250gitlab # [ 105.662851] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.400Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1251gitlab # [ 105.665186] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.403Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1252gitlab # [ 105.667911] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.406Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1253gitlab # [ 105.670346] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.408Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1254gitlab # [ 105.672422] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.410Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1255gitlab # [ 105.674347] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.412Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1256gitlab # [ 105.677208] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.415Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1257gitlab # [ 105.678884] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:11:38.417Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1258gitlab # [ 107.650123] gitlab-workhorse[828]: time="2026-09-18T22:11:40Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1259gitlab # [ 108.509919] sidekiq-cluster[1600]: InitializerConnections Query: SET client_min_messages TO 'warning'1260gitlab # [ 108.510607] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1261gitlab # [ 108.511777] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1262gitlab # [ 108.512929] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1263gitlab # [ 108.514124] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1264gitlab # [ 108.514830] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1265gitlab # [ 108.515952] sidekiq-cluster[1600]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1266gitlab # [ 108.533788] sidekiq-cluster[1600]: InitializerConnections Query: SET standard_conforming_strings = on1267gitlab # [ 108.536762] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1268gitlab # [ 108.539496] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1269gitlab # [ 108.541872] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1270gitlab # [ 108.544613] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1271gitlab # [ 108.546883] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1272gitlab # [ 108.549591] sidekiq-cluster[1600]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1273gitlab # [ 108.552693] sidekiq-cluster[1600]: InitializerConnections Query: SET intervalstyle = iso_86011274gitlab # [ 108.554970] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1275gitlab # [ 108.555101] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1276gitlab # [ 108.555749] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1277gitlab # [ 108.559683] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1278gitlab # [ 108.560841] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1279gitlab # [ 108.563155] sidekiq-cluster[1600]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1280gitlab # [ 108.564794] sidekiq-cluster[1600]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname1281gitlab # [ 108.565049] sidekiq-cluster[1600]: FROM pg_type as t1282gitlab # [ 108.565564] sidekiq-cluster[1600]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')1283gitlab # [ 108.565983] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1284gitlab # [ 108.566613] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1285gitlab # [ 108.567440] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1286gitlab # [ 108.567842] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1287gitlab # [ 108.568475] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1288gitlab # [ 108.569136] sidekiq-cluster[1600]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1289gitlab # [ 108.578989] sidekiq-cluster[1600]: InitializerConnections Query: SET SESSION timezone TO 'UTC'1290gitlab # [ 108.579331] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1291gitlab # [ 108.579469] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1292gitlab # [ 108.579934] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1293gitlab # [ 108.580652] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1294gitlab # [ 108.581229] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1295gitlab # [ 108.581472] sidekiq-cluster[1600]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1296gitlab # [ 108.584605] sidekiq-cluster[1600]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics'1297gitlab # [ 108.585120] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1298gitlab # [ 108.585596] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1299gitlab # [ 108.586065] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1300gitlab # [ 108.586692] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1301gitlab # [ 108.587439] sidekiq-cluster[1600]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1302gitlab # [ 108.587814] sidekiq-cluster[1600]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1303gitlab # [ 117.650919] gitlab-workhorse[828]: time="2026-09-18T22:11:50Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1304gitlab # [ 127.651943] gitlab-workhorse[828]: time="2026-09-18T22:12:00Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1305gitlab # [ 129.037450] postgres[2089]: [2089] ERROR: relation "table_without_models" does not exist1306gitlab # [ 129.038779] postgres[2089]: [2089] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1307gitlab # [ 129.527828] postgres[2100]: [2100] ERROR: relation "table_without_models" does not exist1308gitlab # [ 129.529462] postgres[2100]: [2100] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1309gitlab # [ 131.849446] bundle[1576]: {"timestamp":"2026-09-18T22:12:04.586Z","pid":1576,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1310gitlab # [ 131.854494] bundle[1576]: {"timestamp":"2026-09-18T22:12:04.586Z","pid":1576,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1311gitlab # [ 131.859351] bundle[1576]: {"timestamp":"2026-09-18T22:12:04.586Z","pid":1576,"message":"! #\u003cThread:0x00007689645df728 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1312gitlab # [ 131.872824] bundle[1576]: {"timestamp":"2026-09-18T22:12:04.586Z","pid":1576,"message":"! #\u003cThread:0x000076896485f2a0 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1313gitlab # [ 131.886239] bundle[1576]: {"timestamp":"2026-09-18T22:12:04.587Z","pid":1576,"message":"Use Ctrl-C to stop"}1314gitlab # [ 131.889786] bundle[1576]: {"timestamp":"2026-09-18T22:12:04.587Z","pid":1576,"message":"Sending status to systemd every 1.0 sec"}1315gitlab # [ 132.038442] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:04.772Z","message":"Booted Rails 7.2.3.1 application in production environment"}1316gitlab # [ 132.039163] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:04.776Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1317gitlab # [ 132.039507] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:04.777Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1318gitlab # [ 132.039944] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:04.777Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1319gitlab # [ 132.047078] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:04.779Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1320gitlab # [ 132.225646] bundle[1576]: {"timestamp":"2026-09-18T22:12:04.963Z","pid":1576,"message":"- Worker 0 (PID: 2174) booted in 0.34s, phase: 0"}1321gitlab # [ 132.254542] bundle[1576]: {"timestamp":"2026-09-18T22:12:04.992Z","pid":1576,"message":"- Worker 1 (PID: 2178) booted in 0.34s, phase: 0"}1322gitlab # [ 132.258916] systemd[1]: Started gitlab.service.1323gitlab # [ 132.259929] systemd[1]: Reached target Common target for all GitLab services..1324gitlab # [ 132.263191] systemd[1]: Starting setup-gitlab.service...1325gitlab # [ 132.286414] setup-gitlab-start[2204]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1326gitlab # [ 133.442725] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:06.179Z","message":"Cleaning working queues"}1327gitlab # [ 133.444849] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:06.182Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d7ed4662dcb0f0c69b9edd9f","created_at":"2026-09-18T22:11:02.679Z","trace_propagation_headers":{"sentry-trace":"03d037a900f94fd390e4ba61ed7b00b0-9a98060ac1e94cd7","baggage":"sentry-trace_id=03d037a900f94fd390e4ba61ed7b00b0,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"048f5f067d44d0abafccf5b7dec3ce4f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-18T22:11:02.687Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d7ed4662dcb0f0c69b9edd9f: start","job_status":"start","queue_duration_s":63.495246,"scheduling_latency_s":63.495288}1328gitlab # [ 133.605450] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:06.342Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d7ed4662dcb0f0c69b9edd9f","created_at":"2026-09-18T22:11:02.679Z","trace_propagation_headers":{"sentry-trace":"03d037a900f94fd390e4ba61ed7b00b0-9a98060ac1e94cd7","baggage":"sentry-trace_id=03d037a900f94fd390e4ba61ed7b00b0,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"048f5f067d44d0abafccf5b7dec3ce4f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-18T22:11:02.687Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d7ed4662dcb0f0c69b9edd9f: done: 0.160259 sec","job_status":"done","queue_duration_s":63.495246,"scheduling_latency_s":63.495288,"redis_calls":26,"redis_duration_s":0.0071990000000000005,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003528,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002331,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00134,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.081414259,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.160259,"completed_at":"2026-09-18T22:12:06.342Z","load_balancing_strategy":"primary","db_duration_s":0.047682,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1329gitlab # [ 134.723784] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:12:07 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 24161330gitlab # [ 134.727744] setup-gitlab-start[2204]: + TOKEN=nixbot-test-token-of-20-chars1331gitlab # [ 134.729464] setup-gitlab-start[2204]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1332gitlab # [ 134.740408] runuser[2229]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1333gitlab # [ 135.110098] setup-gitlab-start[2230]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1334gitlab # [ 135.110348] setup-gitlab-start[2230]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1335gitlab # [ 135.111646] setup-gitlab-start[2230]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1336gitlab # [ 135.684452] setup-gitlab-start[2230]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1337gitlab # [ 135.684858] setup-gitlab-start[2230]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1338gitlab # [ 135.686689] setup-gitlab-start[2230]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1339gitlab # [ 147.506797] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.244Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"346a4467a220d488fb7cdb7a","created_at":"2026-09-18T22:12:20.229Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b62feb147cc07e45554ab4e953f0752","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.243Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-346a4467a220d488fb7cdb7a: start","job_status":"start","queue_duration_s":0.00087,"scheduling_latency_s":0.000895}1340gitlab # [ 147.649174] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.386Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"346a4467a220d488fb7cdb7a","created_at":"2026-09-18T22:12:20.229Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b62feb147cc07e45554ab4e953f0752","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.243Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-346a4467a220d488fb7cdb7a: done: 0.142193 sec","job_status":"done","queue_duration_s":0.00087,"scheduling_latency_s":0.000895,"redis_calls":12,"redis_duration_s":0.005639,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001158,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002364,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001080584,"exclusive_lock_hold_duration_s":0.08512199200000001,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.142193,"completed_at":"2026-09-18T22:12:20.386Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030601,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 147.693350] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.431Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"25250cd0e7ef1710135790d7","created_at":"2026-09-18T22:12:20.261Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"651f0584ee7a0c099aa42c796f4b58d2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.267Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-25250cd0e7ef1710135790d7: start","job_status":"start","queue_duration_s":0.163278,"scheduling_latency_s":0.163302}1342gitlab # [ 147.755956] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"25250cd0e7ef1710135790d7","created_at":"2026-09-18T22:12:20.261Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"651f0584ee7a0c099aa42c796f4b58d2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.267Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-25250cd0e7ef1710135790d7: done: 0.062304 sec","job_status":"done","queue_duration_s":0.163278,"scheduling_latency_s":0.163302,"redis_calls":12,"redis_duration_s":0.010436,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001417,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004934,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004085,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000955987,"exclusive_lock_hold_duration_s":0.004026488999999999,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.062304,"completed_at":"2026-09-18T22:12:20.493Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 147.767276] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5adef3f7a934b9f9e28a8044","created_at":"2026-09-18T22:12:20.276Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ccae912fdca31ef59432e7dfc168df5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.312Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5adef3f7a934b9f9e28a8044: start","job_status":"start","queue_duration_s":0.192005,"scheduling_latency_s":0.192046}1344gitlab # [ 147.809623] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.547Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1345gitlab # [ 147.829461] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.566Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5adef3f7a934b9f9e28a8044","created_at":"2026-09-18T22:12:20.276Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ccae912fdca31ef59432e7dfc168df5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.312Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5adef3f7a934b9f9e28a8044: done: 0.062061 sec","job_status":"done","queue_duration_s":0.192005,"scheduling_latency_s":0.192046,"redis_calls":10,"redis_duration_s":0.017338,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000959,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014999,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000417092,"duration_s":0.062061,"completed_at":"2026-09-18T22:12:20.566Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 147.850161] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.571Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"23190e4c0067285eb2366dba","created_at":"2026-09-18T22:12:20.318Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"248f14d641ad117986aedda7c98f5530","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.324Z","job_size_bytes":3,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-23190e4c0067285eb2366dba: start","job_status":"start","queue_duration_s":0.246799,"scheduling_latency_s":0.246833}1347gitlab # [ 147.899664] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.637Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"23190e4c0067285eb2366dba","created_at":"2026-09-18T22:12:20.318Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"248f14d641ad117986aedda7c98f5530","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.324Z","job_size_bytes":3,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-23190e4c0067285eb2366dba: done: 0.065847 sec","job_status":"done","queue_duration_s":0.246799,"scheduling_latency_s":0.246833,"redis_calls":10,"redis_duration_s":0.006199,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000907,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001561,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001519467,"duration_s":0.065847,"completed_at":"2026-09-18T22:12:20.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 147.901769] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5270135da1236ef1aa975f82","created_at":"2026-09-18T22:12:20.329Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1337522c5646d3af2fd95cf3ecc304c9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.333Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5270135da1236ef1aa975f82: start","job_status":"start","queue_duration_s":0.306007,"scheduling_latency_s":0.306039}1349gitlab # [ 147.934114] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.671Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5270135da1236ef1aa975f82","created_at":"2026-09-18T22:12:20.329Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1337522c5646d3af2fd95cf3ecc304c9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.333Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5270135da1236ef1aa975f82: done: 0.032245 sec","job_status":"done","queue_duration_s":0.306007,"scheduling_latency_s":0.306039,"redis_calls":10,"redis_duration_s":0.003066,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000589,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001736,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000741,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000465423,"duration_s":0.032245,"completed_at":"2026-09-18T22:12:20.671Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 147.950092] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"92c484d222faa8d115997564","created_at":"2026-09-18T22:12:20.341Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62aefec54e74098d132575e804058b06","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.354Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-92c484d222faa8d115997564: start","job_status":"start","queue_duration_s":0.318736,"scheduling_latency_s":0.318761}1351gitlab # [ 147.962909] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.683Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1352gitlab # [ 148.001838] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"92c484d222faa8d115997564","created_at":"2026-09-18T22:12:20.341Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62aefec54e74098d132575e804058b06","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.354Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-92c484d222faa8d115997564: done: 0.014402 sec","job_status":"done","queue_duration_s":0.318736,"scheduling_latency_s":0.318761,"redis_calls":10,"redis_duration_s":0.001276,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000244,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000305,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00019807,"duration_s":0.014402,"completed_at":"2026-09-18T22:12:20.687Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 148.003618] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:20.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"966bf38e688a3b8bffebb208","created_at":"2026-09-18T22:12:20.371Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6d834ba886b9a06d734b456cdc1ffe1","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.377Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-966bf38e688a3b8bffebb208: start","job_status":"start","queue_duration_s":0.311326,"scheduling_latency_s":0.311339}1354gitlab # [ 148.331495] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"966bf38e688a3b8bffebb208","created_at":"2026-09-18T22:12:20.371Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6d834ba886b9a06d734b456cdc1ffe1","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.377Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-966bf38e688a3b8bffebb208: done: 0.380304 sec","job_status":"done","queue_duration_s":0.311326,"scheduling_latency_s":0.311339,"redis_calls":565,"redis_duration_s":0.20330099999999998,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000238,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.202031,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.20338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000288584,"duration_s":0.380304,"completed_at":"2026-09-18T22:12:21.069Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 148.397846] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"df47a9a44ed36eedfb53a5f3","created_at":"2026-09-18T22:12:20.387Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e33ca39c24ae96274b8e8fac99aa382","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.447Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-df47a9a44ed36eedfb53a5f3: start","job_status":"start","queue_duration_s":0.688414,"scheduling_latency_s":0.688444}1356gitlab # [ 148.428028] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.165Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1357gitlab # [ 148.442510] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.179Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"df47a9a44ed36eedfb53a5f3","created_at":"2026-09-18T22:12:20.387Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e33ca39c24ae96274b8e8fac99aa382","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.447Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-df47a9a44ed36eedfb53a5f3: done: 0.044123 sec","job_status":"done","queue_duration_s":0.688414,"scheduling_latency_s":0.688444,"redis_calls":10,"redis_duration_s":0.003648,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000476,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000439162,"duration_s":0.044123,"completed_at":"2026-09-18T22:12:21.179Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000543,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 148.480637] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.218Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"3f9ba707cceaa71f9e5b55f8","created_at":"2026-09-18T22:12:20.454Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1da37999e6e56902e6f3d7e479829730","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.467Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-3f9ba707cceaa71f9e5b55f8: start","job_status":"start","queue_duration_s":0.750751,"scheduling_latency_s":0.750783}1359gitlab # [ 148.525659] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.262Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"3f9ba707cceaa71f9e5b55f8","created_at":"2026-09-18T22:12:20.454Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1da37999e6e56902e6f3d7e479829730","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.467Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-3f9ba707cceaa71f9e5b55f8: done: 0.043838 sec","job_status":"done","queue_duration_s":0.750751,"scheduling_latency_s":0.750783,"redis_calls":12,"redis_duration_s":0.004509,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000792,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000694,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001934,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000466539,"duration_s":0.043838,"completed_at":"2026-09-18T22:12:21.262Z","load_balancing_strategy":"primary","db_duration_s":0.000902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 148.526899] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.264Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8c418598ac53a8f30b43ae20","created_at":"2026-09-18T22:12:20.476Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38c482852ca54a13268ea48ffa13f28b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.485Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8c418598ac53a8f30b43ae20: start","job_status":"start","queue_duration_s":0.779069,"scheduling_latency_s":0.7791}1361gitlab # [ 148.592212] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.329Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8c418598ac53a8f30b43ae20","created_at":"2026-09-18T22:12:20.476Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38c482852ca54a13268ea48ffa13f28b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.485Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8c418598ac53a8f30b43ae20: done: 0.064914 sec","job_status":"done","queue_duration_s":0.779069,"scheduling_latency_s":0.7791,"redis_calls":14,"redis_duration_s":0.0042780000000000006,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001392,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001226,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001199315,"exclusive_lock_hold_duration_s":0.028674314,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.064914,"completed_at":"2026-09-18T22:12:21.329Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 148.634505] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.371Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"c7d07428551a6cce7a3ecfe4","created_at":"2026-09-18T22:12:20.501Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7dcc1fd73ba976418655368e1a4c5de","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.511Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-c7d07428551a6cce7a3ecfe4: start","job_status":"start","queue_duration_s":0.860282,"scheduling_latency_s":0.860316}1363gitlab # [ 148.677337] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.414Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"c7d07428551a6cce7a3ecfe4","created_at":"2026-09-18T22:12:20.501Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7dcc1fd73ba976418655368e1a4c5de","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.511Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-c7d07428551a6cce7a3ecfe4: done: 0.043062 sec","job_status":"done","queue_duration_s":0.860282,"scheduling_latency_s":0.860316,"redis_calls":12,"redis_duration_s":0.002458,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000387,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00085,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000875809,"exclusive_lock_hold_duration_s":0.006421207999999999,"duration_s":0.043062,"completed_at":"2026-09-18T22:12:21.414Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002794,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 148.717738] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.455Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"2818add3e4d4bd9ec8caeb5a","created_at":"2026-09-18T22:12:20.537Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"053352af247802b550ea00607306c640","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-18T22:12:20.551Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-2818add3e4d4bd9ec8caeb5a: start","job_status":"start","queue_duration_s":0.904275,"scheduling_latency_s":0.90433}1365gitlab # [ 148.735459] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.473Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1366gitlab # [ 148.743734] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"2818add3e4d4bd9ec8caeb5a","created_at":"2026-09-18T22:12:20.537Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"053352af247802b550ea00607306c640","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-18T22:12:20.551Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-2818add3e4d4bd9ec8caeb5a: done: 0.026462 sec","job_status":"done","queue_duration_s":0.904275,"scheduling_latency_s":0.90433,"redis_calls":10,"redis_duration_s":0.003713,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000634,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355911,"duration_s":0.026462,"completed_at":"2026-09-18T22:12:21.481Z","load_balancing_strategy":"primary","db_duration_s":0.000482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 148.781409] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.519Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9ca7cb0646d2649bb27fd6ef","created_at":"2026-09-18T22:12:20.562Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e997988a648cf15966d3d9497f728f04","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.572Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9ca7cb0646d2649bb27fd6ef: start","job_status":"start","queue_duration_s":0.946462,"scheduling_latency_s":0.946487}1368gitlab # [ 148.993817] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.731Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9ca7cb0646d2649bb27fd6ef","created_at":"2026-09-18T22:12:20.562Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e997988a648cf15966d3d9497f728f04","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.572Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9ca7cb0646d2649bb27fd6ef: done: 0.212324 sec","job_status":"done","queue_duration_s":0.946462,"scheduling_latency_s":0.946487,"redis_calls":14,"redis_duration_s":0.004172,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001256,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00143,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.137858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000787251,"exclusive_lock_hold_duration_s":0.167941176,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.212324,"completed_at":"2026-09-18T22:12:21.731Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.060802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 148.995664] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.732Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"98e7a8b90cd4426db9f81308","created_at":"2026-09-18T22:12:20.619Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0d54f5cc1a0b55cf38911e5cca56db7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.635Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-98e7a8b90cd4426db9f81308: start","job_status":"start","queue_duration_s":1.097393,"scheduling_latency_s":1.097409}1370gitlab # [ 149.119104] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.856Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"98e7a8b90cd4426db9f81308","created_at":"2026-09-18T22:12:20.619Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0d54f5cc1a0b55cf38911e5cca56db7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.635Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-98e7a8b90cd4426db9f81308: done: 0.12367 sec","job_status":"done","queue_duration_s":1.097393,"scheduling_latency_s":1.097409,"redis_calls":16,"redis_duration_s":0.004765,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002725,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001476,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000564,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.088,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000622984,"duration_s":0.12367,"completed_at":"2026-09-18T22:12:21.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.087778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 149.168920] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.906Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4a4addab03b2e9e5483b900a","created_at":"2026-09-18T22:12:20.798Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72243dc7113a2487efc06a381353a961","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.805Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4a4addab03b2e9e5483b900a: start","job_status":"start","queue_duration_s":1.10153,"scheduling_latency_s":1.101561}1372gitlab # [ 149.223519] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:21.960Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4a4addab03b2e9e5483b900a","created_at":"2026-09-18T22:12:20.798Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72243dc7113a2487efc06a381353a961","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.805Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4a4addab03b2e9e5483b900a: done: 0.05419 sec","job_status":"done","queue_duration_s":1.10153,"scheduling_latency_s":1.101561,"redis_calls":10,"redis_duration_s":0.001807,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000337,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000338,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000318197,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.05419,"completed_at":"2026-09-18T22:12:21.960Z","load_balancing_strategy":"primary","db_duration_s":0.013702,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 149.263501] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"bf9907b2bca6edafc0a3709c","created_at":"2026-09-18T22:12:20.828Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec74b617bd24a6e7864e332532d6491f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-18T22:12:20.835Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-bf9907b2bca6edafc0a3709c: start","job_status":"start","queue_duration_s":1.165437,"scheduling_latency_s":1.165479}1374gitlab # [ 149.282171] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.020Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1375gitlab # [ 149.292973] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.030Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"bf9907b2bca6edafc0a3709c","created_at":"2026-09-18T22:12:20.828Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec74b617bd24a6e7864e332532d6491f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-18T22:12:20.835Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-bf9907b2bca6edafc0a3709c: done: 0.029433 sec","job_status":"done","queue_duration_s":1.165437,"scheduling_latency_s":1.165479,"redis_calls":10,"redis_duration_s":0.002484,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001392,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00057,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000423797,"duration_s":0.029433,"completed_at":"2026-09-18T22:12:22.030Z","load_balancing_strategy":"primary","db_duration_s":0.000634,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 149.330505] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.068Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"056be8ba46bbf6ad8a6731d1","created_at":"2026-09-18T22:12:20.841Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.847Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-056be8ba46bbf6ad8a6731d1: start","job_status":"start","queue_duration_s":1.220694,"scheduling_latency_s":1.220732}1377gitlab # [ 149.387452] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.124Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"056be8ba46bbf6ad8a6731d1","created_at":"2026-09-18T22:12:20.841Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.847Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-056be8ba46bbf6ad8a6731d1: done: 0.056865 sec","job_status":"done","queue_duration_s":1.220694,"scheduling_latency_s":1.220732,"redis_calls":19,"redis_duration_s":0.003909,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000869,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000268,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002144,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000628,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000357308,"duration_s":0.056865,"completed_at":"2026-09-18T22:12:22.124Z","load_balancing_strategy":"primary","db_duration_s":0.003116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 149.430393] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.167Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"47734c3181f5e9ffb9daf7f1","created_at":"2026-09-18T22:12:20.858Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27fec1d5c4ec0bee0f26ccc9c9edd45e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.866Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-47734c3181f5e9ffb9daf7f1: start","job_status":"start","queue_duration_s":1.301384,"scheduling_latency_s":1.301448}1379gitlab # [ 149.480465] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"47734c3181f5e9ffb9daf7f1","created_at":"2026-09-18T22:12:20.858Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27fec1d5c4ec0bee0f26ccc9c9edd45e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.866Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-47734c3181f5e9ffb9daf7f1: done: 0.05033 sec","job_status":"done","queue_duration_s":1.301384,"scheduling_latency_s":1.301448,"redis_calls":10,"redis_duration_s":0.0021390000000000003,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000360101,"duration_s":0.05033,"completed_at":"2026-09-18T22:12:22.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 149.521102] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"6418028420beb9931769b521","created_at":"2026-09-18T22:12:20.869Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"888c2be353a80c910e6b09138a764e0e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.896Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-6418028420beb9931769b521: start","job_status":"start","queue_duration_s":1.361711,"scheduling_latency_s":1.361753}1381gitlab # [ 149.564062] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.300Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"6418028420beb9931769b521","created_at":"2026-09-18T22:12:20.869Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"888c2be353a80c910e6b09138a764e0e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.896Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-6418028420beb9931769b521: done: 0.042151 sec","job_status":"done","queue_duration_s":1.361711,"scheduling_latency_s":1.361753,"redis_calls":12,"redis_duration_s":0.003846,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000615,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001878,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001353,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000784458,"exclusive_lock_hold_duration_s":0.009015112,"duration_s":0.042151,"completed_at":"2026-09-18T22:12:22.300Z","load_balancing_strategy":"primary","db_duration_s":0.004238,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 149.565325] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"3b22160df3a74a91f40113c4","created_at":"2026-09-18T22:12:20.907Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7525988eefbd41dd9bfa0f5c373ca23c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.914Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-3b22160df3a74a91f40113c4: start","job_status":"start","queue_duration_s":1.388144,"scheduling_latency_s":1.388177}1383gitlab # [ 149.587672] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.325Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"3b22160df3a74a91f40113c4","created_at":"2026-09-18T22:12:20.907Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7525988eefbd41dd9bfa0f5c373ca23c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.914Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-3b22160df3a74a91f40113c4: done: 0.022448 sec","job_status":"done","queue_duration_s":1.388144,"scheduling_latency_s":1.388177,"redis_calls":12,"redis_duration_s":0.002741,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001364,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000872,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000292216,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.022448,"completed_at":"2026-09-18T22:12:22.325Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 149.614790] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.326Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4d979fc67b5ccd30b162a658","created_at":"2026-09-18T22:12:20.933Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f76b2c973bc2f5993140b8a30c14c894","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.939Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4d979fc67b5ccd30b162a658: start","job_status":"start","queue_duration_s":1.387556,"scheduling_latency_s":1.387572}1385gitlab # [ 149.678245] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"4d979fc67b5ccd30b162a658","created_at":"2026-09-18T22:12:20.933Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f76b2c973bc2f5993140b8a30c14c894","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.939Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-4d979fc67b5ccd30b162a658: done: 0.089026 sec","job_status":"done","queue_duration_s":1.387556,"scheduling_latency_s":1.387572,"redis_calls":12,"redis_duration_s":0.0028309999999999997,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001151,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00037,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.07,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.07,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000297524,"duration_s":0.089026,"completed_at":"2026-09-18T22:12:22.415Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.070445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 149.716254] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c797954353d68a81fb5368e3","created_at":"2026-09-18T22:12:20.944Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a5a6f9474618dd6af9b525fbd7b8ccd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.949Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c797954353d68a81fb5368e3: start","job_status":"start","queue_duration_s":1.504575,"scheduling_latency_s":1.50461}1387gitlab # [ 149.725629] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.463Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1388gitlab # [ 149.730737] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c797954353d68a81fb5368e3","created_at":"2026-09-18T22:12:20.944Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a5a6f9474618dd6af9b525fbd7b8ccd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.949Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c797954353d68a81fb5368e3: done: 0.014646 sec","job_status":"done","queue_duration_s":1.504575,"scheduling_latency_s":1.50461,"redis_calls":10,"redis_duration_s":0.001445,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000295,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028188,"duration_s":0.014646,"completed_at":"2026-09-18T22:12:22.468Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000377,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 149.731844] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"317d4d88cb7bce8798d77f8f","created_at":"2026-09-18T22:12:20.962Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.967Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-317d4d88cb7bce8798d77f8f: start","job_status":"start","queue_duration_s":1.502338,"scheduling_latency_s":1.502351}1390gitlab # [ 149.799569] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"317d4d88cb7bce8798d77f8f","created_at":"2026-09-18T22:12:20.962Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.967Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-317d4d88cb7bce8798d77f8f: done: 0.067814 sec","job_status":"done","queue_duration_s":1.502338,"scheduling_latency_s":1.502351,"redis_calls":71,"redis_duration_s":0.009941,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001326,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000627,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.007598,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00039,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386921,"duration_s":0.067814,"completed_at":"2026-09-18T22:12:22.537Z","load_balancing_strategy":"primary","db_duration_s":0.001195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 149.836779] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.574Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"25d4a0855df4eacdf7e3d87c","created_at":"2026-09-18T22:12:20.971Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e5d1404fcdcd00b14f0403cf8f22841","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.976Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-25d4a0855df4eacdf7e3d87c: start","job_status":"start","queue_duration_s":1.598641,"scheduling_latency_s":1.598664}1392gitlab # [ 149.864701] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.602Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"25d4a0855df4eacdf7e3d87c","created_at":"2026-09-18T22:12:20.971Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e5d1404fcdcd00b14f0403cf8f22841","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.976Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-25d4a0855df4eacdf7e3d87c: done: 0.027787 sec","job_status":"done","queue_duration_s":1.598641,"scheduling_latency_s":1.598664,"redis_calls":10,"redis_duration_s":0.001661,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000359,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208127,"duration_s":0.027787,"completed_at":"2026-09-18T22:12:22.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000477,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 149.901915] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7e77b1669d29237485e61570","created_at":"2026-09-18T22:12:20.980Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b303d8e47e90e9e026fcd0647cf1e84","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.986Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7e77b1669d29237485e61570: start","job_status":"start","queue_duration_s":1.653816,"scheduling_latency_s":1.653838}1394gitlab # [ 149.935379] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7e77b1669d29237485e61570","created_at":"2026-09-18T22:12:20.980Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b303d8e47e90e9e026fcd0647cf1e84","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-18T22:12:20.986Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7e77b1669d29237485e61570: done: 0.033311 sec","job_status":"done","queue_duration_s":1.653816,"scheduling_latency_s":1.653838,"redis_calls":13,"redis_duration_s":0.001867,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000674,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006117816,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.033311,"completed_at":"2026-09-18T22:12:22.673Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002156,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1395gitlab # [ 149.974301] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.712Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"938df9f21f170a73c0aeab36","created_at":"2026-09-18T22:12:20.994Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fef9396fed9bb34a56d187ab825fc837","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-18T22:12:20.999Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-938df9f21f170a73c0aeab36: start","job_status":"start","queue_duration_s":1.712658,"scheduling_latency_s":1.712681}1396gitlab # [ 150.000246] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.738Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"938df9f21f170a73c0aeab36","created_at":"2026-09-18T22:12:20.994Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-1699fc53cf894b2e","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fef9396fed9bb34a56d187ab825fc837","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-18T22:12:20.999Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-938df9f21f170a73c0aeab36: done: 0.025965 sec","job_status":"done","queue_duration_s":1.712658,"scheduling_latency_s":1.712681,"redis_calls":10,"redis_duration_s":0.002235,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001313,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000322946,"duration_s":0.025965,"completed_at":"2026-09-18T22:12:22.738Z","load_balancing_strategy":"primary","db_duration_s":0.004928,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 150.036992] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.774Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e6324e6838f2ab2edeea570b","created_at":"2026-09-18T22:12:22.099Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-235a386523574264","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.115Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e6324e6838f2ab2edeea570b: start","job_status":"start","queue_duration_s":0.658955,"scheduling_latency_s":0.658987}1398gitlab # [ 150.073160] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.811Z","message":"No existing merge request to be cleaned up."}1399gitlab # [ 150.091949] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.829Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e6324e6838f2ab2edeea570b","created_at":"2026-09-18T22:12:22.099Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-235a386523574264","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.115Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e6324e6838f2ab2edeea570b: done: 0.054849 sec","job_status":"done","queue_duration_s":0.658955,"scheduling_latency_s":0.658987,"redis_calls":13,"redis_duration_s":0.006318,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000781,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001661,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003876,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000462908,"duration_s":0.054849,"completed_at":"2026-09-18T22:12:22.829Z","load_balancing_strategy":"primary","db_duration_s":0.004982,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 150.127777] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.865Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6a989f3a10a2fc292634944a","created_at":"2026-09-18T22:12:22.099Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-235a386523574264","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.115Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6a989f3a10a2fc292634944a: start","job_status":"start","queue_duration_s":0.74996,"scheduling_latency_s":0.749997}1401gitlab # [ 150.146493] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.884Z","message":"No existing merge request to be cleaned up."}1402gitlab # [ 150.159155] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.896Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6a989f3a10a2fc292634944a","created_at":"2026-09-18T22:12:22.099Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-235a386523574264","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.115Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6a989f3a10a2fc292634944a: done: 0.03138 sec","job_status":"done","queue_duration_s":0.74996,"scheduling_latency_s":0.749997,"redis_calls":10,"redis_duration_s":0.004791999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001437,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003355,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03138,"completed_at":"2026-09-18T22:12:22.896Z","load_balancing_strategy":"primary","db_duration_s":0.003052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 150.194199] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.931Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"590bd9bdd897981c07435b4f","created_at":"2026-09-18T22:12:22.099Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-235a386523574264","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.115Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-590bd9bdd897981c07435b4f: start","job_status":"start","queue_duration_s":0.81638,"scheduling_latency_s":0.816415}1404gitlab # [ 150.213995] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.950Z","message":"No existing merge request to be cleaned up."}1405gitlab # [ 150.228456] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.965Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"590bd9bdd897981c07435b4f","created_at":"2026-09-18T22:12:22.099Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-235a386523574264","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.115Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-590bd9bdd897981c07435b4f: done: 0.034032 sec","job_status":"done","queue_duration_s":0.81638,"scheduling_latency_s":0.816415,"redis_calls":10,"redis_duration_s":0.007204,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005307,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034032,"completed_at":"2026-09-18T22:12:22.965Z","load_balancing_strategy":"primary","db_duration_s":0.003524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 150.231256] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.969Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b991a27a49b86a5ce5014069","created_at":"2026-09-18T22:12:22.099Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-235a386523574264","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.115Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b991a27a49b86a5ce5014069: start","job_status":"start","queue_duration_s":0.853481,"scheduling_latency_s":0.853509}1407gitlab # [ 150.244962] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.983Z","message":"No existing merge request to be cleaned up."}1408gitlab # [ 150.250873] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.988Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b991a27a49b86a5ce5014069","created_at":"2026-09-18T22:12:22.099Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-235a386523574264","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4eedf6829fed29a2d4e74bf4067cacb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.115Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b991a27a49b86a5ce5014069: done: 0.019823 sec","job_status":"done","queue_duration_s":0.853481,"scheduling_latency_s":0.853509,"redis_calls":10,"redis_duration_s":0.002689,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001794,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019823,"completed_at":"2026-09-18T22:12:22.988Z","load_balancing_strategy":"primary","db_duration_s":0.001455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 150.298096] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:22.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6885592fcb41407e71f88cbc","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6885592fcb41407e71f88cbc: start","job_status":"start","queue_duration_s":0.463702,"scheduling_latency_s":0.46372}1410gitlab # [ 150.298712] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6885592fcb41407e71f88cbc","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6885592fcb41407e71f88cbc: done: 0.017204 sec","job_status":"done","queue_duration_s":0.463702,"scheduling_latency_s":0.46372,"redis_calls":16,"redis_duration_s":0.002744,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00174,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000267352,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017204,"completed_at":"2026-09-18T22:12:23.011Z","load_balancing_strategy":"primary","db_duration_s":0.000482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 150.342937] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ae03866e8620f1e26d05db","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ae03866e8620f1e26d05db: start","job_status":"start","queue_duration_s":0.482107,"scheduling_latency_s":0.48212}1412gitlab # [ 150.343790] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ae03866e8620f1e26d05db","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ae03866e8620f1e26d05db: done: 0.010798 sec","job_status":"done","queue_duration_s":0.482107,"scheduling_latency_s":0.48212,"redis_calls":13,"redis_duration_s":0.001765,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010798,"completed_at":"2026-09-18T22:12:23.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 150.385116] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcd8b70784199d908b8862f0","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcd8b70784199d908b8862f0: start","job_status":"start","queue_duration_s":0.494,"scheduling_latency_s":0.494012}1414gitlab # [ 150.385848] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcd8b70784199d908b8862f0","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcd8b70784199d908b8862f0: done: 0.010613 sec","job_status":"done","queue_duration_s":0.494,"scheduling_latency_s":0.494012,"redis_calls":13,"redis_duration_s":0.001824,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010613,"completed_at":"2026-09-18T22:12:23.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 150.435226] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55db9bdd7c7d0e16fae5c3c2","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55db9bdd7c7d0e16fae5c3c2: start","job_status":"start","queue_duration_s":0.510897,"scheduling_latency_s":0.510914}1416gitlab # [ 150.435862] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55db9bdd7c7d0e16fae5c3c2","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55db9bdd7c7d0e16fae5c3c2: done: 0.011607 sec","job_status":"done","queue_duration_s":0.510897,"scheduling_latency_s":0.510914,"redis_calls":13,"redis_duration_s":0.0018319999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001235,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011607,"completed_at":"2026-09-18T22:12:23.053Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 150.481163] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89b7adb37ecb141bee71be12","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89b7adb37ecb141bee71be12: start","job_status":"start","queue_duration_s":0.523429,"scheduling_latency_s":0.523474}1418gitlab # [ 150.481874] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89b7adb37ecb141bee71be12","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89b7adb37ecb141bee71be12: done: 0.010697 sec","job_status":"done","queue_duration_s":0.523429,"scheduling_latency_s":0.523474,"redis_calls":13,"redis_duration_s":0.00198,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010697,"completed_at":"2026-09-18T22:12:23.064Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 150.490320] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa4fa2696425640a984bc071","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa4fa2696425640a984bc071: start","job_status":"start","queue_duration_s":0.53494,"scheduling_latency_s":0.534954}1420gitlab # [ 150.531363] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa4fa2696425640a984bc071","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa4fa2696425640a984bc071: done: 0.010792 sec","job_status":"done","queue_duration_s":0.53494,"scheduling_latency_s":0.534954,"redis_calls":13,"redis_duration_s":0.001987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001237,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010792,"completed_at":"2026-09-18T22:12:23.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 150.535704] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9ddd00313c0003a63fbc43e","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9ddd00313c0003a63fbc43e: start","job_status":"start","queue_duration_s":0.546681,"scheduling_latency_s":0.546696}1422gitlab # [ 150.581077] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9ddd00313c0003a63fbc43e","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9ddd00313c0003a63fbc43e: done: 0.013529 sec","job_status":"done","queue_duration_s":0.546681,"scheduling_latency_s":0.546696,"redis_calls":13,"redis_duration_s":0.004441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003805,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013529,"completed_at":"2026-09-18T22:12:23.090Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 150.582584] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0997b95204cc55eb65d99929","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0997b95204cc55eb65d99929: start","job_status":"start","queue_duration_s":0.561155,"scheduling_latency_s":0.561169}1424gitlab # [ 150.593354] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0997b95204cc55eb65d99929","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0997b95204cc55eb65d99929: done: 0.010946 sec","job_status":"done","queue_duration_s":0.561155,"scheduling_latency_s":0.561169,"redis_calls":13,"redis_duration_s":0.001797,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010946,"completed_at":"2026-09-18T22:12:23.102Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 150.663294] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f359476e2ef55842507669f","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f359476e2ef55842507669f: start","job_status":"start","queue_duration_s":0.573034,"scheduling_latency_s":0.573047}1426gitlab # [ 150.664080] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f359476e2ef55842507669f","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f359476e2ef55842507669f: done: 0.066236 sec","job_status":"done","queue_duration_s":0.573034,"scheduling_latency_s":0.573047,"redis_calls":13,"redis_duration_s":0.001961,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.066236,"completed_at":"2026-09-18T22:12:23.170Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 150.675547] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68554ea21cfd7c5a7a55ed1a","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68554ea21cfd7c5a7a55ed1a: start","job_status":"start","queue_duration_s":0.640406,"scheduling_latency_s":0.640424}1428gitlab # [ 150.686676] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68554ea21cfd7c5a7a55ed1a","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68554ea21cfd7c5a7a55ed1a: done: 0.015324 sec","job_status":"done","queue_duration_s":0.640406,"scheduling_latency_s":0.640424,"redis_calls":13,"redis_duration_s":0.004706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003489,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015324,"completed_at":"2026-09-18T22:12:23.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 150.734824] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53f7173e23f4a554b6dee312","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53f7173e23f4a554b6dee312: start","job_status":"start","queue_duration_s":0.656687,"scheduling_latency_s":0.656702}1430gitlab # [ 150.735828] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53f7173e23f4a554b6dee312","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53f7173e23f4a554b6dee312: done: 0.010019 sec","job_status":"done","queue_duration_s":0.656687,"scheduling_latency_s":0.656702,"redis_calls":13,"redis_duration_s":0.0018510000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001226,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010019,"completed_at":"2026-09-18T22:12:23.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 150.789379] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee3bdc9e4a5fefff3af2c7c2","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee3bdc9e4a5fefff3af2c7c2: start","job_status":"start","queue_duration_s":0.667705,"scheduling_latency_s":0.667719}1432gitlab # [ 150.790354] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee3bdc9e4a5fefff3af2c7c2","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee3bdc9e4a5fefff3af2c7c2: done: 0.01028 sec","job_status":"done","queue_duration_s":0.667705,"scheduling_latency_s":0.667719,"redis_calls":13,"redis_duration_s":0.0018180000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001208,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01028,"completed_at":"2026-09-18T22:12:23.208Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 150.831706] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397f718253ebfa9e541b570d","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397f718253ebfa9e541b570d: start","job_status":"start","queue_duration_s":0.679253,"scheduling_latency_s":0.679267}1434gitlab # [ 150.832845] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397f718253ebfa9e541b570d","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397f718253ebfa9e541b570d: done: 0.014622 sec","job_status":"done","queue_duration_s":0.679253,"scheduling_latency_s":0.679267,"redis_calls":13,"redis_duration_s":0.005434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004137,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001297,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014622,"completed_at":"2026-09-18T22:12:23.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 150.884916] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"448d7c2c91213517f3501b65","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-448d7c2c91213517f3501b65: start","job_status":"start","queue_duration_s":0.695708,"scheduling_latency_s":0.695723}1436gitlab # [ 150.885881] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"448d7c2c91213517f3501b65","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-448d7c2c91213517f3501b65: done: 0.011222 sec","job_status":"done","queue_duration_s":0.695708,"scheduling_latency_s":0.695723,"redis_calls":13,"redis_duration_s":0.0026420000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001317,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001325,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011222,"completed_at":"2026-09-18T22:12:23.237Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 150.936944] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"211ace0fce8a17e7b773a7f3","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-211ace0fce8a17e7b773a7f3: start","job_status":"start","queue_duration_s":0.707718,"scheduling_latency_s":0.70773}1438gitlab # [ 150.937966] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"211ace0fce8a17e7b773a7f3","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-211ace0fce8a17e7b773a7f3: done: 0.008823 sec","job_status":"done","queue_duration_s":0.707718,"scheduling_latency_s":0.70773,"redis_calls":13,"redis_duration_s":0.0014290000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000455,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008823,"completed_at":"2026-09-18T22:12:23.247Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 150.979883] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f1e18d53122e4c6a4497736","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f1e18d53122e4c6a4497736: start","job_status":"start","queue_duration_s":0.717296,"scheduling_latency_s":0.717309}1440gitlab # [ 150.980898] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f1e18d53122e4c6a4497736","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f1e18d53122e4c6a4497736: done: 0.009489 sec","job_status":"done","queue_duration_s":0.717296,"scheduling_latency_s":0.717309,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009489,"completed_at":"2026-09-18T22:12:23.257Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 150.999941] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"311a19c8d5b380055721ec67","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-311a19c8d5b380055721ec67: start","job_status":"start","queue_duration_s":0.727707,"scheduling_latency_s":0.727721}1442gitlab # [ 151.011760] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"311a19c8d5b380055721ec67","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-311a19c8d5b380055721ec67: done: 0.009685 sec","job_status":"done","queue_duration_s":0.727707,"scheduling_latency_s":0.727721,"redis_calls":13,"redis_duration_s":0.002067,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001403,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009685,"completed_at":"2026-09-18T22:12:23.268Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 151.063444] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16b2e726581d09d08f71517c","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16b2e726581d09d08f71517c: start","job_status":"start","queue_duration_s":0.738327,"scheduling_latency_s":0.738339}1444gitlab # [ 151.064359] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16b2e726581d09d08f71517c","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16b2e726581d09d08f71517c: done: 0.009505 sec","job_status":"done","queue_duration_s":0.738327,"scheduling_latency_s":0.738339,"redis_calls":13,"redis_duration_s":0.001669,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009505,"completed_at":"2026-09-18T22:12:23.278Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 151.093263] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba2524b4eb3615e49db923c0","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba2524b4eb3615e49db923c0: start","job_status":"start","queue_duration_s":0.748643,"scheduling_latency_s":0.748656}1446gitlab # [ 151.133200] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba2524b4eb3615e49db923c0","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba2524b4eb3615e49db923c0: done: 0.009176 sec","job_status":"done","queue_duration_s":0.748643,"scheduling_latency_s":0.748656,"redis_calls":13,"redis_duration_s":0.001515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009176,"completed_at":"2026-09-18T22:12:23.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 151.139378] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d395a8e59376f7e2cb10abf3","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d395a8e59376f7e2cb10abf3: start","job_status":"start","queue_duration_s":0.759063,"scheduling_latency_s":0.759077}1448gitlab # [ 151.150980] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d395a8e59376f7e2cb10abf3","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d395a8e59376f7e2cb10abf3: done: 0.009124 sec","job_status":"done","queue_duration_s":0.759063,"scheduling_latency_s":0.759077,"redis_calls":13,"redis_duration_s":0.001504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009124,"completed_at":"2026-09-18T22:12:23.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 151.186697] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb7e3043c184b2ca6b40359a","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb7e3043c184b2ca6b40359a: start","job_status":"start","queue_duration_s":0.768995,"scheduling_latency_s":0.769008}1450gitlab # [ 151.197882] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb7e3043c184b2ca6b40359a","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb7e3043c184b2ca6b40359a: done: 0.009261 sec","job_status":"done","queue_duration_s":0.768995,"scheduling_latency_s":0.769008,"redis_calls":13,"redis_duration_s":0.001903,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001398,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009261,"completed_at":"2026-09-18T22:12:23.309Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 151.237687] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fb8bcfa63526c9b627ad2e7","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fb8bcfa63526c9b627ad2e7: start","job_status":"start","queue_duration_s":0.779078,"scheduling_latency_s":0.779091}1452gitlab # [ 151.244626] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fb8bcfa63526c9b627ad2e7","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fb8bcfa63526c9b627ad2e7: done: 0.009153 sec","job_status":"done","queue_duration_s":0.779078,"scheduling_latency_s":0.779091,"redis_calls":13,"redis_duration_s":0.0014949999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009153,"completed_at":"2026-09-18T22:12:23.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 151.293094] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfc3ca3b14795f330fb850a0","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfc3ca3b14795f330fb850a0: start","job_status":"start","queue_duration_s":0.789041,"scheduling_latency_s":0.789055}1454gitlab # [ 151.293747] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfc3ca3b14795f330fb850a0","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfc3ca3b14795f330fb850a0: done: 0.009929 sec","job_status":"done","queue_duration_s":0.789041,"scheduling_latency_s":0.789055,"redis_calls":13,"redis_duration_s":0.001479,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009929,"completed_at":"2026-09-18T22:12:23.329Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 151.338584] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e17a5ee3d5e7049a08339ab","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e17a5ee3d5e7049a08339ab: start","job_status":"start","queue_duration_s":0.799966,"scheduling_latency_s":0.799979}1456gitlab # [ 151.339422] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e17a5ee3d5e7049a08339ab","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e17a5ee3d5e7049a08339ab: done: 0.009568 sec","job_status":"done","queue_duration_s":0.799966,"scheduling_latency_s":0.799979,"redis_calls":13,"redis_duration_s":0.001589,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009568,"completed_at":"2026-09-18T22:12:23.340Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 151.392203] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a0850c9f4eb5b446a903905","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a0850c9f4eb5b446a903905: start","job_status":"start","queue_duration_s":0.810352,"scheduling_latency_s":0.810365}1458gitlab # [ 151.392871] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a0850c9f4eb5b446a903905","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a0850c9f4eb5b446a903905: done: 0.009222 sec","job_status":"done","queue_duration_s":0.810352,"scheduling_latency_s":0.810365,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009222,"completed_at":"2026-09-18T22:12:23.350Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 151.431800] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b84c20b691d75ffd1f896bd","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b84c20b691d75ffd1f896bd: start","job_status":"start","queue_duration_s":0.820366,"scheduling_latency_s":0.820379}1460gitlab # [ 151.432639] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b84c20b691d75ffd1f896bd","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b84c20b691d75ffd1f896bd: done: 0.009232 sec","job_status":"done","queue_duration_s":0.820366,"scheduling_latency_s":0.820379,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009232,"completed_at":"2026-09-18T22:12:23.360Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 151.471669] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16284a27d5a9f8240192e378","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16284a27d5a9f8240192e378: start","job_status":"start","queue_duration_s":0.83077,"scheduling_latency_s":0.830783}1462gitlab # [ 151.472565] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16284a27d5a9f8240192e378","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16284a27d5a9f8240192e378: done: 0.009244 sec","job_status":"done","queue_duration_s":0.83077,"scheduling_latency_s":0.830783,"redis_calls":13,"redis_duration_s":0.001514,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009244,"completed_at":"2026-09-18T22:12:23.370Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 151.511092] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"754e4bb58503881663654528","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-754e4bb58503881663654528: start","job_status":"start","queue_duration_s":0.840905,"scheduling_latency_s":0.840923}1464gitlab # [ 151.511483] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"754e4bb58503881663654528","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-754e4bb58503881663654528: done: 0.009629 sec","job_status":"done","queue_duration_s":0.840905,"scheduling_latency_s":0.840923,"redis_calls":13,"redis_duration_s":0.002177,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001681,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009629,"completed_at":"2026-09-18T22:12:23.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 151.560507] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00234dfb73e8afbea48c5275","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00234dfb73e8afbea48c5275: start","job_status":"start","queue_duration_s":0.851332,"scheduling_latency_s":0.851345}1466gitlab # [ 151.561756] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00234dfb73e8afbea48c5275","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00234dfb73e8afbea48c5275: done: 0.009201 sec","job_status":"done","queue_duration_s":0.851332,"scheduling_latency_s":0.851345,"redis_calls":13,"redis_duration_s":0.001522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009201,"completed_at":"2026-09-18T22:12:23.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 151.591267] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c69942bcdc335a005414011e","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c69942bcdc335a005414011e: start","job_status":"start","queue_duration_s":0.86134,"scheduling_latency_s":0.861354}1468gitlab # [ 151.630521] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c69942bcdc335a005414011e","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c69942bcdc335a005414011e: done: 0.009336 sec","job_status":"done","queue_duration_s":0.86134,"scheduling_latency_s":0.861354,"redis_calls":13,"redis_duration_s":0.001537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009336,"completed_at":"2026-09-18T22:12:23.401Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 151.634435] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1649a2e3be7081b3b88a8c5a","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1649a2e3be7081b3b88a8c5a: start","job_status":"start","queue_duration_s":0.871987,"scheduling_latency_s":0.872001}1470gitlab # [ 151.672065] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1649a2e3be7081b3b88a8c5a","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1649a2e3be7081b3b88a8c5a: done: 0.009367 sec","job_status":"done","queue_duration_s":0.871987,"scheduling_latency_s":0.872001,"redis_calls":13,"redis_duration_s":0.0015110000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009367,"completed_at":"2026-09-18T22:12:23.412Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 151.678083] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b0d12d51f2e87510a5af6c7","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b0d12d51f2e87510a5af6c7: start","job_status":"start","queue_duration_s":0.88218,"scheduling_latency_s":0.882193}1472gitlab # [ 151.688916] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b0d12d51f2e87510a5af6c7","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b0d12d51f2e87510a5af6c7: done: 0.009391 sec","job_status":"done","queue_duration_s":0.88218,"scheduling_latency_s":0.882193,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009391,"completed_at":"2026-09-18T22:12:23.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 151.733700] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5709a98e0e990c951939e896","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5709a98e0e990c951939e896: start","job_status":"start","queue_duration_s":0.892368,"scheduling_latency_s":0.892381}1474gitlab # [ 151.734649] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5709a98e0e990c951939e896","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5709a98e0e990c951939e896: done: 0.0093 sec","job_status":"done","queue_duration_s":0.892368,"scheduling_latency_s":0.892381,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0093,"completed_at":"2026-09-18T22:12:23.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 151.783691] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52c66f8b0b4c48f636f0ccec","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52c66f8b0b4c48f636f0ccec: start","job_status":"start","queue_duration_s":0.902496,"scheduling_latency_s":0.902509}1476gitlab # [ 151.784417] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52c66f8b0b4c48f636f0ccec","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52c66f8b0b4c48f636f0ccec: done: 0.009405 sec","job_status":"done","queue_duration_s":0.902496,"scheduling_latency_s":0.902509,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009405,"completed_at":"2026-09-18T22:12:23.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 151.811672] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5b5ac6e395f7aa7d390df15","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5b5ac6e395f7aa7d390df15: start","job_status":"start","queue_duration_s":0.913131,"scheduling_latency_s":0.913146}1478gitlab # [ 151.823197] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5b5ac6e395f7aa7d390df15","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5b5ac6e395f7aa7d390df15: done: 0.009673 sec","job_status":"done","queue_duration_s":0.913131,"scheduling_latency_s":0.913146,"redis_calls":13,"redis_duration_s":0.001524,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009673,"completed_at":"2026-09-18T22:12:23.453Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 151.868778] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7dd71268fe218720b8c63ae","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7dd71268fe218720b8c63ae: start","job_status":"start","queue_duration_s":0.923308,"scheduling_latency_s":0.923322}1480gitlab # [ 151.869818] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7dd71268fe218720b8c63ae","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7dd71268fe218720b8c63ae: done: 0.00908 sec","job_status":"done","queue_duration_s":0.923308,"scheduling_latency_s":0.923322,"redis_calls":13,"redis_duration_s":0.001809,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001322,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00908,"completed_at":"2026-09-18T22:12:23.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 151.912309] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2b1274a7973c80e8dbfd211","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2b1274a7973c80e8dbfd211: start","job_status":"start","queue_duration_s":0.933203,"scheduling_latency_s":0.933216}1482gitlab # [ 151.914627] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2b1274a7973c80e8dbfd211","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2b1274a7973c80e8dbfd211: done: 0.009178 sec","job_status":"done","queue_duration_s":0.933203,"scheduling_latency_s":0.933216,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009178,"completed_at":"2026-09-18T22:12:23.473Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 151.956765] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa0b06d589ce7775a3042c4d","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa0b06d589ce7775a3042c4d: start","job_status":"start","queue_duration_s":0.943203,"scheduling_latency_s":0.943217}1484gitlab # [ 151.958746] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa0b06d589ce7775a3042c4d","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa0b06d589ce7775a3042c4d: done: 0.009435 sec","job_status":"done","queue_duration_s":0.943203,"scheduling_latency_s":0.943217,"redis_calls":13,"redis_duration_s":0.001685,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009435,"completed_at":"2026-09-18T22:12:23.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 152.005758] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30bf3d26f75beeea85759b32","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30bf3d26f75beeea85759b32: start","job_status":"start","queue_duration_s":0.953837,"scheduling_latency_s":0.953851}1486gitlab # [ 152.006603] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30bf3d26f75beeea85759b32","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30bf3d26f75beeea85759b32: done: 0.009118 sec","job_status":"done","queue_duration_s":0.953837,"scheduling_latency_s":0.953851,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009118,"completed_at":"2026-09-18T22:12:23.493Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 152.050870] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3838b383a5091c799b9dccf","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3838b383a5091c799b9dccf: start","job_status":"start","queue_duration_s":0.963871,"scheduling_latency_s":0.963884}1488gitlab # [ 152.051769] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3838b383a5091c799b9dccf","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3838b383a5091c799b9dccf: done: 0.009179 sec","job_status":"done","queue_duration_s":0.963871,"scheduling_latency_s":0.963884,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009179,"completed_at":"2026-09-18T22:12:23.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 152.084062] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8b733f9553b6a18c6435af3","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8b733f9553b6a18c6435af3: start","job_status":"start","queue_duration_s":0.973886,"scheduling_latency_s":0.973899}1490gitlab # [ 152.131842] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8b733f9553b6a18c6435af3","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8b733f9553b6a18c6435af3: done: 0.009317 sec","job_status":"done","queue_duration_s":0.973886,"scheduling_latency_s":0.973899,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009317,"completed_at":"2026-09-18T22:12:23.513Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 152.132956] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c0842d3130271361493444e","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c0842d3130271361493444e: start","job_status":"start","queue_duration_s":0.984036,"scheduling_latency_s":0.984049}1492gitlab # [ 152.176784] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c0842d3130271361493444e","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c0842d3130271361493444e: done: 0.009091 sec","job_status":"done","queue_duration_s":0.984036,"scheduling_latency_s":0.984049,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009091,"completed_at":"2026-09-18T22:12:23.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 152.178326] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7c3eaa0df90fd4e906d2ed1","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7c3eaa0df90fd4e906d2ed1: start","job_status":"start","queue_duration_s":0.994364,"scheduling_latency_s":0.994376}1494gitlab # [ 152.221821] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7c3eaa0df90fd4e906d2ed1","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7c3eaa0df90fd4e906d2ed1: done: 0.009239 sec","job_status":"done","queue_duration_s":0.994364,"scheduling_latency_s":0.994376,"redis_calls":13,"redis_duration_s":0.001539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009239,"completed_at":"2026-09-18T22:12:23.534Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 152.224234] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11c0c8f204e2ee59946c87c6","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11c0c8f204e2ee59946c87c6: start","job_status":"start","queue_duration_s":1.004507,"scheduling_latency_s":1.00452}1496gitlab # [ 152.270137] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11c0c8f204e2ee59946c87c6","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11c0c8f204e2ee59946c87c6: done: 0.009031 sec","job_status":"done","queue_duration_s":1.004507,"scheduling_latency_s":1.00452,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009031,"completed_at":"2026-09-18T22:12:23.544Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 152.271691] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee8b90d345c35d3850cd2308","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee8b90d345c35d3850cd2308: start","job_status":"start","queue_duration_s":1.014336,"scheduling_latency_s":1.014349}1498gitlab # [ 152.318094] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee8b90d345c35d3850cd2308","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee8b90d345c35d3850cd2308: done: 0.009049 sec","job_status":"done","queue_duration_s":1.014336,"scheduling_latency_s":1.014349,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009049,"completed_at":"2026-09-18T22:12:23.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 152.319676] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b7065015e6930bb84db71b4","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b7065015e6930bb84db71b4: start","job_status":"start","queue_duration_s":1.024595,"scheduling_latency_s":1.024608}1500gitlab # [ 152.330895] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b7065015e6930bb84db71b4","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b7065015e6930bb84db71b4: done: 0.009127 sec","job_status":"done","queue_duration_s":1.024595,"scheduling_latency_s":1.024608,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009127,"completed_at":"2026-09-18T22:12:23.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 152.385525] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bb5dd7b04223a68975488e2","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bb5dd7b04223a68975488e2: start","job_status":"start","queue_duration_s":1.034538,"scheduling_latency_s":1.03455}1502gitlab # [ 152.386411] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bb5dd7b04223a68975488e2","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bb5dd7b04223a68975488e2: done: 0.009431 sec","job_status":"done","queue_duration_s":1.034538,"scheduling_latency_s":1.03455,"redis_calls":13,"redis_duration_s":0.0015600000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009431,"completed_at":"2026-09-18T22:12:23.574Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 152.441304] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"def90fab54311ece242252e8","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-def90fab54311ece242252e8: start","job_status":"start","queue_duration_s":1.044789,"scheduling_latency_s":1.044801}1504gitlab # [ 152.442257] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"def90fab54311ece242252e8","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-def90fab54311ece242252e8: done: 0.009307 sec","job_status":"done","queue_duration_s":1.044789,"scheduling_latency_s":1.044801,"redis_calls":13,"redis_duration_s":0.001725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009307,"completed_at":"2026-09-18T22:12:23.584Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 152.481954] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4081b49bce027a6d37cb6c27","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4081b49bce027a6d37cb6c27: start","job_status":"start","queue_duration_s":1.054916,"scheduling_latency_s":1.054929}1506gitlab # [ 152.482766] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4081b49bce027a6d37cb6c27","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4081b49bce027a6d37cb6c27: done: 0.009094 sec","job_status":"done","queue_duration_s":1.054916,"scheduling_latency_s":1.054929,"redis_calls":13,"redis_duration_s":0.001609,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009094,"completed_at":"2026-09-18T22:12:23.594Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 152.523463] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5247b0c14d0f3e01f87b0641","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5247b0c14d0f3e01f87b0641: start","job_status":"start","queue_duration_s":1.064924,"scheduling_latency_s":1.064937}1508gitlab # [ 152.524762] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:23.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5247b0c14d0f3e01f87b0641","created_at":"2026-09-18T22:12:22.481Z","trace_propagation_headers":{"sentry-trace":"270ab9401cd44e9a9e3bf6e9720f9db4-884af1df9fe54542","baggage":"sentry-trace_id=270ab9401cd44e9a9e3bf6e9720f9db4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5e5f1b629a49733bbf7be77cdbbd84d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:12:22.530Z","job_size_bytes":2,"pid":1600,"sidekiq_tid":"madw","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5247b0c14d0f3e01f87b0641: done: 0.009818 sec","job_status":"done","queue_duration_s":1.064924,"scheduling_latency_s":1.064937,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009818,"completed_at":"2026-09-18T22:12:23.605Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 152.743328] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2332994240 wd_nsec: 23329942651510gitlab # [ 161.933928] setup-gitlab-start[2230]: W, [2026-09-18T22:12:34.671383 #2230] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1511gitlab # [ 183.694606] postgres[2435]: [2435] ERROR: relation "table_without_models" does not exist1512gitlab # [ 183.694878] postgres[2435]: [2435] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1513gitlab # [ 186.642133] runuser[2229]: pam_unix(runuser:session): session closed for user gitlab1514gitlab # [ 186.643830] setup-gitlab-start[2204]: + systemctl restart gitlab-sidekiq.service1515gitlab # [ 186.659938] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:59.397Z","message":"Shutting down"}1516gitlab # [ 186.660462] systemd[1]: Stopping gitlab-sidekiq.service...1517gitlab # [ 186.660927] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:59.397Z","message":"Scheduler exiting..."}1518gitlab # [ 186.661436] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:59.398Z","message":"Terminating quiet threads for default capsule"}1519gitlab # [ 186.664594] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:59.402Z","message":"Scheduler exiting..."}1520gitlab # [ 187.165690] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:12:59.903Z","message":"Pausing to allow jobs to finish..."}1521gitlab # [ 191.169651] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:13:03.906Z","message":"Bye!"}1522gitlab # [ 191.184280] sidekiq-cluster[1600]: {"severity":"INFO","time":"2026-09-18T22:13:03.922Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1600,"worker_id":"sidekiq_0","memwd_rss_bytes":940449792,"retry":0}1523gitlab # [ 191.478256] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1524gitlab # [ 191.479174] systemd[1]: Stopped gitlab-sidekiq.service.1525gitlab # [ 191.479633] systemd[1]: gitlab-sidekiq.service: Consumed 1min 2.311s CPU time over 2min 1.098s wall clock time, 957.2M memory peak, 4.2M read from disk.1526gitlab # [ 191.485292] systemd[1]: Started gitlab-sidekiq.service.1527gitlab # [ 191.486532] setup-gitlab-start[2204]: + mkdir -p /var/lib/secrets1528gitlab # [ 191.487293] setup-gitlab-start[2204]: + echo nixbot-test-token-of-20-chars1529gitlab # [ 191.487864] setup-gitlab-start[2204]: + chmod 644 /var/lib/secrets/gitlab-token1530gitlab # [ 191.488766] setup-gitlab-start[2204]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1531gitlab # [ 191.648605] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:04 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 1361532gitlab # [ 192.432673] gitaly[895]: time="2026-09-18T22:13:05.170Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.170" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1533gitlab # [ 192.441792] gitaly[895]: time="2026-09-18T22:13:05.170Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.170" grpc.time_ms=0.249 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1534gitlab # [ 192.501860] gitaly[895]: time="2026-09-18T22:13:05.239Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.079" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.239" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1535gitlab # [ 192.511086] gitaly[895]: time="2026-09-18T22:13:05.240Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.079" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.239" grpc.time_ms=0.291 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1536gitlab # [ 192.558194] gitaly[895]: time="2026-09-18T22:13:05.296Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH diskcache=fce6febe-ffc4-4d34-a9be-b415064570fe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-18T22:13:15.099" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.259" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1537gitlab # [ 192.558984] gitaly[895]: time="2026-09-18T22:13:05.296Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1579 command.oublock=32 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-18T22:13:15.099" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.259" grpc.time_ms=36.913 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1538gitlab # [ 192.595470] gitaly[895]: time="2026-09-18T22:13:05.333Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.172" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.332" pid=895 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1539gitlab # [ 192.596262] gitaly[895]: time="2026-09-18T22:13:05.333Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.172" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.332" grpc.time_ms=0.672 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1540gitlab # [ 192.627514] gitaly[895]: time="2026-09-18T22:13:05.365Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-18T22:13:15.197" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.357" grpc.time_ms=7.682 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1541gitlab # [ 192.693345] gitaly[895]: time="2026-09-18T22:13:05.431Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.270" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.431" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1542gitlab # [ 192.702628] gitaly[895]: time="2026-09-18T22:13:05.431Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.270" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.431" grpc.time_ms=0.215 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1543gitlab # [ 192.753080] gitaly[895]: time="2026-09-18T22:13:05.491Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH diskcache=08f992c8-5cdf-498c-95db-045e10430e77 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-18T22:13:15.292" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.452" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1544gitlab # [ 192.753487] gitaly[895]: time="2026-09-18T22:13:05.491Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1587 command.oublock=32 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-18T22:13:15.292" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.452" grpc.time_ms=39.039 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1545gitlab # [ 192.760028] gitaly[895]: time="2026-09-18T22:13:05.498Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.497" pid=895 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1546gitlab # [ 192.760818] gitaly[895]: time="2026-09-18T22:13:05.498Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.497" grpc.time_ms=0.778 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1547gitlab # [ 192.770026] gitaly[895]: time="2026-09-18T22:13:05.508Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-18T22:13:15.011" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.501" grpc.time_ms=6.151 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1548gitlab # [ 192.792055] gitaly[895]: time="2026-09-18T22:13:05.530Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:13:15.039" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.529" grpc.time_ms=0.187 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1549gitlab # [ 192.797327] gitaly[895]: time="2026-09-18T22:13:05.535Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-18T22:13:15.044" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.534" grpc.time_ms=0.284 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1550gitlab # [ 193.212820] gitaly[895]: time="2026-09-18T22:13:05.950Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V993RF1FKVFPAJEJE5N3XH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-18T22:13:15.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:05.942" grpc.time_ms=8.529 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1551gitlab # [ 193.604548] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:06 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 19421552gitlab # [ 193.608946] setup-gitlab-start[2490]: {"id":1,"message":null,"error":null}1553gitlab # [ 194.550984] sidekiq-cluster[2478]: {"severity":"INFO","time":"2026-09-18T22:13:07.272Z","message":"Starting cluster with 1 processes"}1554gitlab # [ 198.706450] gitaly[895]: time="2026-09-18T22:13:11.443Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AJ6ET8HXQKN51XA6FRQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-18T22:13:21.277" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:11.438" grpc.time_ms=5.48 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1555gitlab # [ 198.776937] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:11 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1481556gitlab # [ 198.779844] setup-gitlab-start[2204]: + mkdir -p /tmp/test-flake1557gitlab # [ 198.781198] setup-gitlab-start[2204]: + cd /tmp/test-flake1558gitlab # [ 198.781987] setup-gitlab-start[2204]: + git init -b master1559gitlab # [ 198.794927] setup-gitlab-start[2566]: Initialized empty Git repository in /tmp/test-flake/.git/1560gitlab # [ 198.796159] setup-gitlab-start[2204]: + git config user.name test1561gitlab # [ 198.797094] setup-gitlab-start[2204]: + git config user.email test@example.com1562gitlab # [ 198.798343] setup-gitlab-start[2204]: + cat1563gitlab # [ 198.799802] setup-gitlab-start[2204]: + git add flake.nix1564gitlab # [ 198.801896] setup-gitlab-start[2204]: + git commit -m 'initial commit'1565gitlab # [ 198.812446] setup-gitlab-start[2571]: [master (root-commit) 6057835] initial commit1566gitlab # [ 198.812735] setup-gitlab-start[2571]: 1 file changed, 10 insertions(+)1567gitlab # [ 198.813211] setup-gitlab-start[2571]: create mode 100644 flake.nix1568gitlab # [ 198.813787] setup-gitlab-start[2204]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1569gitlab # [ 198.819103] setup-gitlab-start[2204]: + git push -u origin master1570gitlab # [ 198.859575] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:11 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 261571gitlab # [ 198.935770] gitaly[895]: time="2026-09-18T22:13:11.673Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V99ASGH6PZPH9EW6GYAE6W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:11.668" grpc.time_ms=5.735 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1572gitlab # [ 198.948310] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:11 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 861573gitlab # [ 199.242634] gitaly[895]: time="2026-09-18T22:13:11.980Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-18T22:13:21.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:11.970" grpc.time_ms=10.261 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1574gitlab # [ 199.313108] gitaly[895]: time="2026-09-18T22:13:12.051Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=326 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-18T22:13:42.041" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-18T22:13:12.041" grpc.time_ms=9.566 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1575gitlab # [ 199.385737] gitaly[895]: time="2026-09-18T22:13:12.123Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS duration_ms=346ns method=POST pid=895 status=200 url="http://unix/api/v4/internal/allowed"1576gitlab # [ 199.418697] gitaly[895]: time="2026-09-18T22:13:12.156Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS duration_ms=32ns method=POST pid=895 status=200 url="http://unix/api/v4/internal/pre_receive"1577gitlab # [ 199.425367] gitaly[895]: time="2026-09-18T22:13:12.163Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:11.776" grpc.time_ms=387.083 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1578gitlab # [ 199.491555] gitaly[895]: time="2026-09-18T22:13:12.229Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:12.228" grpc.time_ms=0.724 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1579gitlab # [ 199.543735] gitaly[895]: time="2026-09-18T22:13:12.281Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:12.281" grpc.time_ms=0.369 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1580gitlab # [ 199.571551] gitaly[895]: time="2026-09-18T22:13:12.309Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:12.308" grpc.time_ms=0.382 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1581gitlab # [ 199.658327] gitaly[895]: time="2026-09-18T22:13:12.396Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-18T22:13:22.225" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:12.386" grpc.time_ms=9.936 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1582gitlab # [ 199.682871] gitaly[895]: time="2026-09-18T22:13:12.420Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1560 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-18T22:13:22.244" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:12.404" grpc.time_ms=16.326 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 199.721423] gitaly[895]: time="2026-09-18T22:13:12.458Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS duration_ms=124ns method=POST pid=895 status=200 url="http://unix/api/v4/internal/post_receive"1584gitlab # [ 199.723622] gitaly[895]: time="2026-09-18T22:13:12.459Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:12.333" grpc.time_ms=125.793 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1585gitlab # [ 199.747605] gitaly[895]: time="2026-09-18T22:13:12.485Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS diskcache=96c74742-317d-4f6e-96ae-43502d4c75da grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:11.739" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1586gitlab # [ 199.748303] gitaly[895]: time="2026-09-18T22:13:12.485Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=146 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=872448 command.minflt=13519 command.oublock=32 command.real_time_ms=738 command.spawn_token_fork_ms=0 command.system_time_ms=94 command.user_time_ms=52 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:11.739" grpc.time_ms=745.825 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1587gitlab # [ 199.749061] gitlab-workhorse[828]: time="2026-09-18T22:13:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521588gitlab # [ 199.749799] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:12 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7921589gitlab # [ 199.772812] setup-gitlab-start[2575]: To http://gitlab/root/test-flake.git1590gitlab # [ 199.773926] setup-gitlab-start[2575]: * [new branch] master -> master1591gitlab # [ 199.776382] setup-gitlab-start[2575]: branch 'master' set up to track 'origin/master'.1592gitlab # [ 199.779381] systemd[1]: Finished setup-gitlab.service.1593gitlab # [ 199.781820] systemd[1]: Starting nixbot CI service...1594gitlab: (finished: waiting for unit setup-gitlab.service, in 201.20 seconds)1595gitlab: waiting for unit nixbot.service1596gitlab # [ 200.747028] nixbot[2660]: {"ts": "2026-09-18T22:13:13.484Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1597gitlab # [ 200.767694] nixbot[2660]: {"ts": "2026-09-18T22:13:13.505Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1598gitlab # [ 200.779741] nixbot[2660]: {"ts": "2026-09-18T22:13:13.517Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1599gitlab # [ 200.782695] nixbot[2660]: {"ts": "2026-09-18T22:13:13.520Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1600gitlab # [ 200.785340] nixbot[2660]: {"ts": "2026-09-18T22:13:13.523Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1601gitlab # [ 200.788894] nixbot[2660]: {"ts": "2026-09-18T22:13:13.527Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1602gitlab # [ 200.791195] nixbot[2660]: {"ts": "2026-09-18T22:13:13.529Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1603gitlab # [ 200.793316] nixbot[2660]: {"ts": "2026-09-18T22:13:13.530Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1604gitlab # [ 200.796730] nixbot[2660]: {"ts": "2026-09-18T22:13:13.534Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1605gitlab # [ 200.799901] nixbot[2660]: {"ts": "2026-09-18T22:13:13.538Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1606gitlab # [ 200.805029] nixbot[2660]: {"ts": "2026-09-18T22:13:13.543Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1607gitlab # [ 200.810402] nixbot[2660]: {"ts": "2026-09-18T22:13:13.548Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1608gitlab # [ 200.814269] nixbot[2660]: {"ts": "2026-09-18T22:13:13.552Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1609gitlab # [ 200.815820] nixbot[2660]: {"ts": "2026-09-18T22:13:13.553Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1610gitlab # [ 200.820435] nixbot[2660]: {"ts": "2026-09-18T22:13:13.558Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1611gitlab # [ 200.823583] nixbot[2660]: {"ts": "2026-09-18T22:13:13.561Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1612gitlab # [ 200.826721] nixbot[2660]: {"ts": "2026-09-18T22:13:13.564Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1613gitlab # [ 200.828208] nixbot[2660]: {"ts": "2026-09-18T22:13:13.566Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1614gitlab # [ 200.833669] nixbot[2660]: {"ts": "2026-09-18T22:13:13.571Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1615gitlab # [ 200.837102] nixbot[2660]: {"ts": "2026-09-18T22:13:13.575Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1616gitlab # [ 200.841357] nixbot[2660]: {"ts": "2026-09-18T22:13:13.579Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1617gitlab # [ 200.847305] nixbot[2660]: {"ts": "2026-09-18T22:13:13.585Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1618gitlab # [ 200.850258] nixbot[2660]: {"ts": "2026-09-18T22:13:13.588Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1619gitlab # [ 200.853810] nixbot[2660]: {"ts": "2026-09-18T22:13:13.591Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1620gitlab # [ 200.857362] nixbot[2660]: {"ts": "2026-09-18T22:13:13.595Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1621gitlab # [ 200.861313] nixbot[2660]: {"ts": "2026-09-18T22:13:13.599Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1622gitlab # [ 200.865088] nixbot[2660]: {"ts": "2026-09-18T22:13:13.602Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1623gitlab # [ 200.868848] nixbot[2660]: {"ts": "2026-09-18T22:13:13.606Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1624gitlab # [ 200.872546] nixbot[2660]: {"ts": "2026-09-18T22:13:13.610Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1625gitlab # [ 200.888719] nixbot[2660]: {"ts": "2026-09-18T22:13:13.626Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1626gitlab # [ 200.894912] nixbot[2660]: {"ts": "2026-09-18T22:13:13.632Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1627gitlab # [ 200.897971] nixbot[2660]: {"ts": "2026-09-18T22:13:13.635Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1628gitlab # [ 200.901628] nixbot[2660]: {"ts": "2026-09-18T22:13:13.639Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1629gitlab # [ 200.906509] nixbot[2660]: {"ts": "2026-09-18T22:13:13.644Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1630gitlab # [ 200.910778] nixbot[2660]: {"ts": "2026-09-18T22:13:13.648Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1631gitlab # [ 200.919633] nixbot[2660]: {"ts": "2026-09-18T22:13:13.657Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1632gitlab # [ 200.925133] nixbot[2660]: {"ts": "2026-09-18T22:13:13.663Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1633gitlab # [ 201.246615] nixbot[2660]: INFO: Started server process [2660]1634gitlab # [ 201.246822] nixbot[2660]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1635gitlab # [ 201.263100] nixbot[2660]: INFO: ::ffff:127.0.0.1:37644 - "GET /health HTTP/1.1" 200 OK1636gitlab # [ 201.267328] systemd[1]: Started nixbot CI service.1637gitlab # [ 201.267484] systemd[1]: Reached target Multi-User System.1638gitlab # [ 201.270708] systemd[1]: Startup finished in 1.172s (kernel) + 3.269s (initrd) + 3min 16.828s (userspace) = 3min 21.270s.1639gitlab: (finished: waiting for unit nixbot.service, in 1.05 seconds)1640gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1641gitlab # [ 201.670130] nixbot[2660]: INFO: ::ffff:127.0.0.1:37658 - "GET /health HTTP/1.1" 200 OK1642gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1643subtest: project discovered and webhook registered1644gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1645gitlab # [ 201.752619] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 461646gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.08 seconds)1647gitlab # [ 201.843445] nixbot[2660]: {"ts": "2026-09-18T22:13:14.580Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1648gitlab # [ 201.849204] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:14 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 5771649gitlab # [ 201.858536] nixbot[2660]: {"ts": "2026-09-18T22:13:14.596Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1650gitlab # [ 201.934399] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 671651gitlab # [ 201.940330] nixbot[2660]: {"ts": "2026-09-18T22:13:14.678Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1652gitlab # [ 201.947449] nixbot[2660]: {"ts": "2026-09-18T22:13:14.685Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1653gitlab # [ 202.057533] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 1021654gitlab # [ 202.063160] nixbot[2660]: {"ts": "2026-09-18T22:13:14.801Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1655gitlab # [ 202.308481] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2331656gitlab # [ 202.309755] nixbot[2660]: {"ts": "2026-09-18T22:13:15.047Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1657gitlab # [ 202.402157] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 821658gitlab # [ 202.407124] nixbot[2660]: {"ts": "2026-09-18T22:13:15.141Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1659gitlab # [ 202.454833] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 461660gitlab # [ 202.459852] nixbot[2660]: {"ts": "2026-09-18T22:13:15.193Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1661gitlab # [ 202.531816] gitaly[895]: time="2026-09-18T22:13:15.269Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99E9XNXGJPFHRDAYJ6EB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-18T22:13:45.255" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:15.255" grpc.time_ms=13.007 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1662gitlab # [ 202.566669] gitaly[895]: time="2026-09-18T22:13:15.304Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99E9XNXGJPFHRDAYJ6EB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:13:45.303" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:15.303" grpc.time_ms=0.764 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1663gitlab # [ 202.600338] gitaly[895]: time="2026-09-18T22:13:15.338Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99E9XNXGJPFHRDAYJ6EB6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-18T22:13:25.001" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:15.331" grpc.time_ms=6.751 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1664gitlab # [ 202.620303] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1611665gitlab # [ 202.622499] nixbot[2660]: {"ts": "2026-09-18T22:13:15.359Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1666gitlab # [ 202.687350] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 631667gitlab # [ 202.688690] nixbot[2660]: {"ts": "2026-09-18T22:13:15.426Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1668gitlab # [ 202.692378] nixbot[2660]: {"ts": "2026-09-18T22:13:15.430Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1669gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1670gitlab # [ 202.792470] gitaly[895]: time="2026-09-18T22:13:15.530Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99EJNXX7VV3AWHVNJ70YX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.519" grpc.time_ms=10.665 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1671gitlab # [ 202.794262] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 531672gitlab # [ 202.838878] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 561673gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.08 seconds)1674(finished: subtest: project discovered and webhook registered, in 1.17 seconds)1675subtest: push delivers the webhook1676gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1677gitlab # [ 202.962331] gitaly[895]: time="2026-09-18T22:13:15.700Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99EMD3S1EGR9N15JJZE37 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.693" pid=895 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1678gitlab # [ 202.984070] gitaly[895]: time="2026-09-18T22:13:15.700Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99EMD3S1EGR9N15JJZE37 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.693" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1679gitlab # [ 203.008678] gitaly[895]: time="2026-09-18T22:13:15.746Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99EMD3S1EGR9N15JJZE37 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.693" grpc.time_ms=53.168 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1680gitlab # [ 203.009647] gitlab-workhorse[828]: time="2026-09-18T22:13:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131681gitlab # [ 203.010795] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 2131682gitlab # [ 203.092871] gitaly[895]: time="2026-09-18T22:13:15.830Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99EV903XJGM9C9SMQQ6C0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.821" pid=895 remote_ip="2001:db8:1::1" request_sha=7b891267182ad4d84ffba921f23a7df6e5427d9b response_bytes=223 span.kind=server system=grpc1683gitlab # [ 203.093300] gitaly[895]: time="2026-09-18T22:13:15.831Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1212 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99EV903XJGM9C9SMQQ6C0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.821" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1684gitlab # [ 203.093941] gitaly[895]: time="2026-09-18T22:13:15.831Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1212 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99EV903XJGM9C9SMQQ6C0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.821" grpc.time_ms=9.821 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1685gitlab # [ 203.105202] gitlab-workhorse[828]: time="2026-09-18T22:13:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231686gitlab # [ 203.105821] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 901687gitlab # [ 203.191092] gitaly[895]: time="2026-09-18T22:13:15.929Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99EZG4B9GDY09E6HP4ZB1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.923" grpc.time_ms=5.7 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1688gitlab # [ 203.191786] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 411689gitlab # [ 203.234257] gitaly[895]: time="2026-09-18T22:13:15.972Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F0TXXS5EM400KPM8RKG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.966" pid=895 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1690gitlab # [ 203.234653] gitaly[895]: time="2026-09-18T22:13:15.972Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F0TXXS5EM400KPM8RKG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.966" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1691gitlab # [ 203.254482] gitaly[895]: time="2026-09-18T22:13:15.992Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F0TXXS5EM400KPM8RKG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:15.966" grpc.time_ms=25.466 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1692gitlab # [ 203.266966] gitlab-workhorse[828]: time="2026-09-18T22:13:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671693gitlab # [ 203.267411] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 741694gitlab # [ 203.347591] gitaly[895]: time="2026-09-18T22:13:16.085Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99F4AGCKBCGQJK1TVK1RF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:16.079" grpc.time_ms=5.215 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1695gitlab # [ 203.348026] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 431696gitlab # [ 203.405992] gitaly[895]: time="2026-09-18T22:13:16.144Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F63E2J2D5BMG01VYSPZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:16.135" pid=895 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1697gitlab # [ 203.406741] gitaly[895]: time="2026-09-18T22:13:16.144Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F63E2J2D5BMG01VYSPZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:16.135" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1698gitlab # [ 203.407547] gitaly[895]: time="2026-09-18T22:13:16.144Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F63E2J2D5BMG01VYSPZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:16.135" grpc.time_ms=9.379 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1699gitlab # [ 203.439275] gitlab-workhorse[828]: time="2026-09-18T22:13:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091700gitlab # [ 203.439800] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 771701gitlab # [ 203.512330] gitaly[895]: time="2026-09-18T22:13:16.250Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:13:46.249" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:16.249" grpc.time_ms=0.631 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1702gitlab # [ 203.558447] gitaly[895]: time="2026-09-18T22:13:16.296Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-18T22:13:46.288" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:16.288" grpc.time_ms=7.954 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1703gitlab # [ 203.583573] gitaly[895]: time="2026-09-18T22:13:16.321Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:13:26.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:16.316" grpc.time_ms=5.263 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1704gitlab # [ 203.605971] gitaly[895]: time="2026-09-18T22:13:16.344Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:13:26.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:16.338" grpc.time_ms=5.104 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1705gitlab # [ 203.678290] gitaly[895]: time="2026-09-18T22:13:16.416Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-18T22:13:26.085" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:16.415" grpc.time_ms=0.34 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1706gitlab # [ 203.697841] gitaly[895]: time="2026-09-18T22:13:16.435Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=619 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-18T22:13:26.100" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:16.430" grpc.time_ms=5.306 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1707gitlab # [ 203.735645] gitaly[895]: time="2026-09-18T22:13:16.473Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:16.473" grpc.time_ms=0.289 pid=895 span.kind=server system=grpc1708gitlab # [ 203.760198] gitaly[895]: time="2026-09-18T22:13:16.498Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:16.497" grpc.time_ms=0.367 pid=895 span.kind=server system=grpc1709gitlab # [ 203.774809] gitaly[895]: time="2026-09-18T22:13:16.512Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP diskcache=6d5a6bd2-13cd-4df6-9552-25806e57218f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:13:26.119" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:16.449" pid=895 ref=refs/keep-around/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b remote_ip="2001:db8:1::1" revision=6057835189e4eae24eaf6d1c6c6f0a70eb7d655b span.kind=server system=grpc username=root1710gitlab # [ 203.775253] gitaly[895]: time="2026-09-18T22:13:16.513Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=53 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5538 command.oublock=8 command.real_time_ms=58 command.spawn_token_fork_ms=0 command.system_time_ms=37 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:13:26.119" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:16.449" grpc.time_ms=63.77 pid=895 ref=refs/keep-around/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b remote_ip="2001:db8:1::1" revision=6057835189e4eae24eaf6d1c6c6f0a70eb7d655b span.kind=server system=grpc username=root1711gitlab # [ 203.800167] gitaly[895]: time="2026-09-18T22:13:16.538Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:13:46.537" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:16.537" grpc.time_ms=0.241 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1712gitlab # [ 203.912808] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 4401713gitlab # [ 203.919562] nixbot[2660]: {"ts": "2026-09-18T22:13:16.651Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b \"HTTP/1.1 201 Created\""}1714gitlab # [ 203.926368] nixbot[2660]: {"ts": "2026-09-18T22:13:16.656Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-6057835189e4-0"}1715gitlab # [ 204.045447] nixbot[2660]: {"ts": "2026-09-18T22:13:16.783Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-6057835189e4-0", "argv0": "bwrap"}1716gitlab # [ 204.142283] systemd[1]: Started Nix Daemon.1717gitlab # [ 204.177351] nix-daemon[2820]: accepted connection from pid 2815, user nixbot1718gitlab # [ 208.899570] nix-daemon[2820]: accepted connection from pid 2863, user nixbot1719gitlab # [ 208.912708] nix-daemon[2820]: accepted connection from pid 2815, user nixbot1720gitlab # [ 213.646728] nix-daemon[2820]: accepted connection from pid 2815, user nixbot1721gitlab # [ 213.728105] nix-daemon[2820]: accepted connection from pid 2912, user nixbot1722gitlab # [ 213.785082] gitaly[895]: time="2026-09-18T22:13:26.522Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99S9S93TBGCQT5ZE17Y37 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:13:56.520" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:26.521" grpc.time_ms=0.934 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1723gitlab # [ 213.831453] gitaly[895]: time="2026-09-18T22:13:26.569Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99S9S93TBGCQT5ZE17Y37 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:13:36.233" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:26.563" grpc.time_ms=6.164 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1724gitlab # [ 213.867273] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1471725gitlab # [ 213.868915] nixbot[2660]: {"ts": "2026-09-18T22:13:26.606Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b \"HTTP/1.1 201 Created\""}1726gitlab # [ 213.913733] gitaly[895]: time="2026-09-18T22:13:26.651Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99SEQD5XAAC80C27KBJJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:13:56.650" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:26.650" grpc.time_ms=0.647 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1727gitlab # [ 213.940590] gitaly[895]: time="2026-09-18T22:13:26.678Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V99SEQD5XAAC80C27KBJJX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:13:36.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:26.671" grpc.time_ms=6.07 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1728gitlab # [ 213.974377] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 961729gitlab # [ 213.975636] nixbot[2660]: {"ts": "2026-09-18T22:13:26.713Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b \"HTTP/1.1 201 Created\""}1730gitlab # [ 218.354699] nix-daemon[2820]: accepted connection from pid 2986, user nixbot1731gitlab # [ 218.429856] gitaly[895]: time="2026-09-18T22:13:31.167Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99XVAB3NWY2P40TG8FPRX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:01.166" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:31.166" grpc.time_ms=0.608 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1732gitlab # [ 218.467546] gitaly[895]: time="2026-09-18T22:13:31.205Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99XVAB3NWY2P40TG8FPRX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:13:41.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:31.200" grpc.time_ms=5.421 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1733gitlab # [ 218.525437] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1471734gitlab # [ 218.525859] nixbot[2660]: {"ts": "2026-09-18T22:13:31.263Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b \"HTTP/1.1 201 Created\""}1735gitlab # [ 218.872079] gitaly[895]: time="2026-09-18T22:13:31.609Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99Y9DDKW7C4CQAFZ6KJEN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:31.599" grpc.time_ms=10.096 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1736gitlab # [ 218.872924] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 451737gitlab # [ 219.013174] gitaly[895]: time="2026-09-18T22:13:31.751Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99YAW3P7ED5KCYQWQXCQW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:31.739" pid=895 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1738gitlab # [ 219.013455] gitaly[895]: time="2026-09-18T22:13:31.751Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99YAW3P7ED5KCYQWQXCQW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:31.739" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1739gitlab # [ 219.014600] gitaly[895]: time="2026-09-18T22:13:31.751Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99YAW3P7ED5KCYQWQXCQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:31.739" grpc.time_ms=12.211 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1740gitlab # [ 219.015813] gitlab-workhorse[828]: time="2026-09-18T22:13:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671741gitlab # [ 219.016859] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1401742gitlab # [ 219.140430] gitaly[895]: time="2026-09-18T22:13:31.878Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V99YHSRN2TZTVE4MV0C3KA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:31.870" grpc.time_ms=8.192 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1743gitlab # [ 219.140852] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 451744gitlab # [ 219.278273] gitaly[895]: time="2026-09-18T22:13:32.016Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99YK80MXQJGAQ3NMF2CSN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:32.004" pid=895 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1745gitlab # [ 219.278539] gitaly[895]: time="2026-09-18T22:13:32.016Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99YK80MXQJGAQ3NMF2CSN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:32.004" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1746gitlab # [ 219.279627] gitaly[895]: time="2026-09-18T22:13:32.016Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99YK80MXQJGAQ3NMF2CSN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:32.004" grpc.time_ms=12.361 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1747gitlab # [ 219.339606] gitlab-workhorse[828]: time="2026-09-18T22:13:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671748gitlab # [ 219.340148] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1971749gitlab # [ 219.529820] sidekiq-cluster[2539]: 2026-09-18T22:13:32.267Z pid=2539 tid=xtv INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2539, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431357952}1750gitlab # [ 219.715710] sidekiq-cluster[2539]: W, [2026-09-18T22:13:32.453656 #2539] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1751gitlab # [ 219.752956] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:32.490Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1752gitlab # [ 221.498328] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:34.235Z","message":"GitLab reliable fetch activated!"}1753gitlab # [ 221.498694] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:34.235Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1754gitlab # [ 241.596790] postgres[3219]: [3219] ERROR: relation "table_without_models" does not exist1755gitlab # [ 241.597107] postgres[3219]: [3219] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1756gitlab # [ 244.240783] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:56.978Z","message":"Booted Rails 7.2.3.1 application in production environment"}1757gitlab # [ 244.245059] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:56.983Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1758gitlab # [ 244.247069] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:56.983Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1759gitlab # [ 244.248719] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:56.983Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1760gitlab # [ 244.250636] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:56.984Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1761gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 42.26 seconds)1762gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1763gitlab # [ 245.186498] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:57 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 401764gitlab # [ 245.251176] gitaly[895]: time="2026-09-18T22:13:57.988Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR17RTJT53F41W9A1RFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:57.982" grpc.time_ms=6.16 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1765gitlab # [ 245.263397] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:58 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 741766gitlab # Enumerating objects: 5, done.1767gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1768gitlab # Delta compression using up to 4 threads1769gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1770gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done.1771gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1772gitlab # [ 245.503647] gitaly[895]: time="2026-09-18T22:13:58.241Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-18T22:14:28.229" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-18T22:13:58.229" grpc.time_ms=11.5 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773gitlab # [ 245.540262] gitaly[895]: time="2026-09-18T22:13:58.278Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2V9AR43XCZ1HAN176AP1RSR duration_ms=115ns method=POST pid=895 status=200 url="http://unix/api/v4/internal/allowed"1774gitlab # [ 245.578527] gitaly[895]: time="2026-09-18T22:13:58.316Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2V9AR43XCZ1HAN176AP1RSR duration_ms=32ns method=POST pid=895 status=200 url="http://unix/api/v4/internal/pre_receive"1775gitlab # [ 245.579313] gitaly[895]: time="2026-09-18T22:13:58.316Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:58.162" grpc.time_ms=154.631 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1776gitlab # [ 245.631140] gitaly[895]: time="2026-09-18T22:13:58.369Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:58.368" grpc.time_ms=0.395 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1777gitlab # [ 245.667160] gitaly[895]: time="2026-09-18T22:13:58.405Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:58.404" grpc.time_ms=0.384 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1778gitlab # [ 245.703715] gitaly[895]: time="2026-09-18T22:13:58.441Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:58.441" grpc.time_ms=0.329 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1779gitlab # [ 245.725769] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:58.462Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7f79f7f7e6aa57cf6f55d611","created_at":"2026-09-18T22:13:05.378Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.414Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7f79f7f7e6aa57cf6f55d611: start","job_status":"start","queue_duration_s":53.047486,"scheduling_latency_s":53.047532}1780gitlab # [ 245.766994] gitaly[895]: time="2026-09-18T22:13:58.504Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2V9AR43XCZ1HAN176AP1RSR duration_ms=36ns method=POST pid=895 status=200 url="http://unix/api/v4/internal/post_receive"1781gitlab # [ 245.767957] gitaly[895]: time="2026-09-18T22:13:58.505Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:58.467" grpc.time_ms=37.872 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1782gitlab # [ 245.787701] gitaly[895]: time="2026-09-18T22:13:58.525Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR diskcache=15673f66-a783-4b45-bbed-fff4d3071e84 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:58.092" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1783gitlab # To http://gitlab/root/test-flake.git1784gitlab # [ 245.797040] gitaly[895]: time="2026-09-18T22:13:58.526Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=184 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=876544 command.minflt=13536 command.oublock=32 command.real_time_ms=426 command.spawn_token_fork_ms=0 command.system_time_ms=113 command.user_time_ms=71 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:13:58.092" grpc.time_ms=433.641 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1785gitlab # [ 245.797922] gitlab-workhorse[828]: time="2026-09-18T22:13:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521786gitlab # [ 245.798490] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:13:58 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 5101787gitlab # 6057835..789e09c master -> master1788gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.72 seconds)1789gitlab: must succeed: git -C /tmp/test-flake rev-parse master1790gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1791gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131'1792gitlab # [ 245.870089] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:58.607Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7f79f7f7e6aa57cf6f55d611","created_at":"2026-09-18T22:13:05.378Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.414Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7f79f7f7e6aa57cf6f55d611: done: 0.145454 sec","job_status":"done","queue_duration_s":53.047486,"scheduling_latency_s":53.047532,"redis_calls":25,"redis_duration_s":0.0064730000000000005,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.002909,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002294,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00127,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.071271907,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.145454,"completed_at":"2026-09-18T22:13:58.607Z","load_balancing_strategy":"primary","db_duration_s":0.039618,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1793gitlab # [ 245.895785] nixbot[2660]: INFO: ::ffff:127.0.0.1:42238 - "GET /api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 200 OK1794gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131', in 0.09 seconds)1795gitlab # [ 245.921221] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:58.658Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"bfe9035d9942fdbd75a944a8","created_at":"2026-09-18T22:13:05.423Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.427Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-bfe9035d9942fdbd75a944a8: start","job_status":"start","queue_duration_s":53.231061,"scheduling_latency_s":53.23111}1796gitlab # [ 246.015451] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:58.753Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"bfe9035d9942fdbd75a944a8","created_at":"2026-09-18T22:13:05.423Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.427Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-bfe9035d9942fdbd75a944a8: done: 0.094512 sec","job_status":"done","queue_duration_s":53.231061,"scheduling_latency_s":53.23111,"redis_calls":14,"redis_duration_s":0.0031,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000579,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00142,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001101,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06690152,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.094512,"completed_at":"2026-09-18T22:13:58.753Z","load_balancing_strategy":"primary","db_duration_s":0.020821,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1797gitlab # [ 246.016805] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:58.754Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"f6075e18c400ee5e6f3c8404","created_at":"2026-09-18T22:13:05.846Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.851Z","job_size_bytes":5,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-f6075e18c400ee5e6f3c8404: start","job_status":"start","queue_duration_s":52.902839,"scheduling_latency_s":52.902853}1798gitlab # [ 246.133712] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:58.871Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"f6075e18c400ee5e6f3c8404","created_at":"2026-09-18T22:13:05.846Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.851Z","job_size_bytes":5,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-f6075e18c400ee5e6f3c8404: done: 0.116811 sec","job_status":"done","queue_duration_s":52.902839,"scheduling_latency_s":52.902853,"redis_calls":12,"redis_duration_s":0.002893,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000944,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001531,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000284394,"duration_s":0.116811,"completed_at":"2026-09-18T22:13:58.871Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024245,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1799gitlab # [ 246.135547] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:58.872Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c12d429eca581b71a98d55bc","created_at":"2026-09-18T22:13:05.872Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.875Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c12d429eca581b71a98d55bc: start","job_status":"start","queue_duration_s":52.996911,"scheduling_latency_s":52.996925}1800gitlab # [ 246.308402] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:59.045Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c12d429eca581b71a98d55bc","created_at":"2026-09-18T22:13:05.872Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.875Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c12d429eca581b71a98d55bc: done: 0.173206 sec","job_status":"done","queue_duration_s":52.996911,"scheduling_latency_s":52.996925,"redis_calls":11,"redis_duration_s":0.002343,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000653,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001283,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.148,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.148,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000295848,"duration_s":0.173206,"completed_at":"2026-09-18T22:13:59.045Z","load_balancing_strategy":"primary","db_duration_s":0.069306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1801gitlab # [ 246.309776] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:59.047Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"a4ce123d4cc9240e17077b3d","created_at":"2026-09-18T22:13:05.877Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.880Z","job_size_bytes":89,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-a4ce123d4cc9240e17077b3d: start","job_status":"start","queue_duration_s":53.166429,"scheduling_latency_s":53.166445}1802gitlab # [ 246.384475] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:59.074Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"a4ce123d4cc9240e17077b3d","created_at":"2026-09-18T22:13:05.877Z","trace_propagation_headers":{"sentry-trace":"f423f859c3f845afb3ed35fe4eea2841-a8a24b3d2fcc4880","baggage":"sentry-trace_id=f423f859c3f845afb3ed35fe4eea2841,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V993RF1FKVFPAJEJE5N3XH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:05.880Z","job_size_bytes":89,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-a4ce123d4cc9240e17077b3d: done: 0.027078 sec","job_status":"done","queue_duration_s":53.166429,"scheduling_latency_s":53.166445,"redis_calls":11,"redis_duration_s":0.002173,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000574,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001214,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000284115,"duration_s":0.027078,"completed_at":"2026-09-18T22:13:59.074Z","load_balancing_strategy":"primary","db_duration_s":0.005028,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1803gitlab # [ 246.398979] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:59.075Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 6057835189e4eae24eaf6d1c6c6f0a70eb7d655b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"606b514fb6c04416216aca86","created_at":"2026-09-18T22:13:12.370Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-befc88d82edf4e0b","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:13:12.373Z","job_size_bytes":185,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-606b514fb6c04416216aca86: start","job_status":"start","queue_duration_s":46.701682,"scheduling_latency_s":46.701698}1804gitlab # [ 246.444099] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:13:59.087Z","message":"changes: \"0000000000000000000000000000000000000000 6057835189e4eae24eaf6d1c6c6f0a70eb7d655b refs/heads/master\\n\""}1805gitlab # [ 246.445122] gitaly[895]: time="2026-09-18T22:13:59.119Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:09.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:59.109" grpc.time_ms=10.647 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806gitlab # [ 246.445790] gitaly[895]: time="2026-09-18T22:13:59.133Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-18T22:14:09.023" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:59.123" grpc.time_ms=9.988 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807gitlab # [ 246.446414] gitaly[895]: time="2026-09-18T22:13:59.177Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-18T22:14:09.069" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:59.170" grpc.time_ms=7.336 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808gitlab # [ 246.484390] gitaly[895]: time="2026-09-18T22:13:59.222Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3931 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:59.222" grpc.time_ms=0.302 pid=895 span.kind=server system=grpc1809gitlab # [ 246.516644] gitaly[895]: time="2026-09-18T22:13:59.254Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:13:59.254" grpc.time_ms=0.459 pid=895 span.kind=server system=grpc1810gitlab # [ 246.534585] gitaly[895]: time="2026-09-18T22:13:59.272Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS diskcache=dd363a52-1c3d-4cf0-b9bd-8a8fb39502f5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:14:09.092" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:59.192" pid=895 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1811gitlab # [ 246.544263] gitaly[895]: time="2026-09-18T22:13:59.282Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=68 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4985 command.oublock=8 command.real_time_ms=72 command.spawn_token_fork_ms=0 command.system_time_ms=50 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:14:09.092" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:59.192" grpc.time_ms=80.024 pid=895 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1812gitlab # [ 246.564819] gitaly[895]: time="2026-09-18T22:13:59.302Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-18T22:14:09.202" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:59.302" grpc.time_ms=0.123 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab # [ 246.579838] gitaly[895]: time="2026-09-18T22:13:59.317Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=938 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-18T22:14:09.208" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:59.309" grpc.time_ms=8.417 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814gitlab # [ 246.805335] gitaly[895]: time="2026-09-18T22:13:59.543Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=340 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-18T22:14:29.038" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:59.538" grpc.time_ms=4.644 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1815gitlab # [ 246.840852] gitaly[895]: time="2026-09-18T22:13:59.578Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-18T22:14:09.008" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:13:59.578" grpc.time_ms=0.222 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1816gitlab # [ 246.858607] gitaly[895]: time="2026-09-18T22:13:59.596Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-18T22:14:09.019" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-18T22:13:59.589" grpc.time_ms=6.907 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab # [ 246.893760] gitaly[895]: time="2026-09-18T22:13:59.631Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:09.052" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:59.623" grpc.time_ms=8.685 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131'1819gitlab # [ 246.912848] gitaly[895]: time="2026-09-18T22:13:59.650Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:09.073" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:13:59.644" grpc.time_ms=6.81 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1820gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131', in 0.03 seconds)1821gitlab # [ 246.935043] gitaly[895]: time="2026-09-18T22:13:59.672Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-18T22:14:29.161" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:59.661" grpc.time_ms=11.162 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1822gitlab # [ 246.945770] nixbot[2660]: INFO: ::ffff:127.0.0.1:42254 - "GET /api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 200 OK1823gitlab # [ 247.251415] gitaly[895]: time="2026-09-18T22:13:59.989Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-18T22:14:09.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:13:59.987" grpc.time_ms=1.884 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1824gitlab # [ 247.278299] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.015Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 6057835189e4eae24eaf6d1c6c6f0a70eb7d655b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"606b514fb6c04416216aca86","created_at":"2026-09-18T22:13:12.370Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-befc88d82edf4e0b","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:13:12.373Z","job_size_bytes":185,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-606b514fb6c04416216aca86: done: 0.939757 sec","job_status":"done","queue_duration_s":46.701682,"scheduling_latency_s":46.701698,"gitaly_calls":13,"gitaly_duration_s":0.23866,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.037277000000000005,"redis_read_bytes":2103,"redis_write_bytes":21745,"redis_cache_calls":15,"redis_cache_duration_s":0.003072,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.004939,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001314,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7833,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002984,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.017448,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.00752,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.171,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.335,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.049,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.475906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000445028,"exclusive_lock_hold_duration_s":0.025702149,"duration_s":0.939757,"completed_at":"2026-09-18T22:14:00.015Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.228249,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1825gitlab # [ 247.293594] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.016Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ebdfd0880e2aac3444f2614d","created_at":"2026-09-18T22:13:16.553Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-9651bc872c234319","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:16.559Z","job_size_bytes":65,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ebdfd0880e2aac3444f2614d: start","job_status":"start","queue_duration_s":43.457137,"scheduling_latency_s":43.457152}1826gitlab # [ 247.368268] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.105Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ebdfd0880e2aac3444f2614d","created_at":"2026-09-18T22:13:16.553Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-9651bc872c234319","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:16.559Z","job_size_bytes":65,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ebdfd0880e2aac3444f2614d: done: 0.089455 sec","job_status":"done","queue_duration_s":43.457137,"scheduling_latency_s":43.457152,"redis_calls":16,"redis_duration_s":0.0021939999999999998,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001083,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000844,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000237739,"duration_s":0.089455,"completed_at":"2026-09-18T22:14:00.105Z","load_balancing_strategy":"primary","db_duration_s":0.049775,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1827gitlab # [ 247.389111] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.107Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"49341c81275a878224809e2e","created_at":"2026-09-18T22:13:16.560Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-9651bc872c234319","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:16.563Z","job_size_bytes":65,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-49341c81275a878224809e2e: start","job_status":"start","queue_duration_s":43.544043,"scheduling_latency_s":43.544058}1828gitlab # [ 247.429032] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.166Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"49341c81275a878224809e2e","created_at":"2026-09-18T22:13:16.560Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-9651bc872c234319","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:16.563Z","job_size_bytes":65,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-49341c81275a878224809e2e: done: 0.059693 sec","job_status":"done","queue_duration_s":43.544043,"scheduling_latency_s":43.544058,"redis_calls":21,"redis_duration_s":0.002762,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.001263,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000794,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000705,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227124,"duration_s":0.059693,"completed_at":"2026-09-18T22:14:00.166Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1829gitlab # [ 247.430099] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.167Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4ec40f1a8e7e52be5c83d665","created_at":"2026-09-18T22:13:16.635Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-9651bc872c234319","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:16.639Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4ec40f1a8e7e52be5c83d665: start","job_status":"start","queue_duration_s":43.528875,"scheduling_latency_s":43.528893}1830gitlab # [ 247.619278] gitaly[895]: time="2026-09-18T22:14:00.356Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:00.356" grpc.time_ms=0.274 pid=895 span.kind=server system=grpc1831gitlab # [ 247.676501] gitaly[895]: time="2026-09-18T22:14:00.414Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:00.413" grpc.time_ms=1.392 pid=895 span.kind=server system=grpc1832gitlab # [ 247.728694] gitaly[895]: time="2026-09-18T22:14:00.466Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:00.464" grpc.time_ms=1.621 pid=895 span.kind=server system=grpc1833gitlab # [ 247.780523] gitaly[895]: time="2026-09-18T22:14:00.518Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP diskcache=43e788b7-70e0-4b62-a822-c0fe09333df6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-18T22:14:30.021" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:00.322" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1834gitlab # [ 247.802942] gitaly[895]: time="2026-09-18T22:14:00.540Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=155 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7441 command.oublock=0 command.real_time_ms=162 command.spawn_token_fork_ms=0 command.system_time_ms=104 command.user_time_ms=51 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-18T22:14:30.021" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:00.322" grpc.time_ms=196.533 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131'1836gitlab # [ 247.960751] gitaly[895]: time="2026-09-18T22:14:00.698Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-18T22:14:30.196" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:00.697" grpc.time_ms=1.509 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1837gitlab # [ 248.009676] gitaly[895]: time="2026-09-18T22:14:00.747Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99F9H7ZQXSYMHK58302RP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:30.245" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:00.746" grpc.time_ms=1.19 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 248.027677] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.765Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4ec40f1a8e7e52be5c83d665","created_at":"2026-09-18T22:13:16.635Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-9651bc872c234319","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:16.639Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2539,"job_status":"rescheduled","message":"PipelineProcessWorker JID-4ec40f1a8e7e52be5c83d665: rescheduled"}1839gitlab # [ 248.032160] nixbot[2660]: INFO: ::ffff:127.0.0.1:42264 - "GET /api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 200 OK1840gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131', in 0.10 seconds)1841gitlab # [ 248.055075] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.792Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4ec40f1a8e7e52be5c83d665","created_at":"2026-09-18T22:13:16.635Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-9651bc872c234319","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:16.639Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4ec40f1a8e7e52be5c83d665: done: 0.624499 sec","job_status":"done","queue_duration_s":43.528875,"scheduling_latency_s":43.528893,"gitaly_calls":3,"gitaly_duration_s":0.230818,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.04404,"redis_read_bytes":460,"redis_write_bytes":29294,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000772,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001742,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.010689,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.005314,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15581,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.020641,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000596,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000929,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002585,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.012,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.014,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.289003,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.5594931750000001,"duration_s":0.624499,"completed_at":"2026-09-18T22:14:00.792Z","load_balancing_strategy":"primary","db_duration_s":0.070923,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1842gitlab # [ 248.104985] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.842Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","6057835189e4eae24eaf6d1c6c6f0a70eb7d655b 789e09c2c59589392a122d7de0d22c17cad48131 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"aae87a137938470b5916d494","created_at":"2026-09-18T22:13:58.491Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-58adf9f3ce064c98","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:13:58.493Z","job_size_bytes":185,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-aae87a137938470b5916d494: start","job_status":"start","queue_duration_s":2.349418,"scheduling_latency_s":2.349461}1843gitlab # [ 248.138306] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.876Z","message":"changes: \"6057835189e4eae24eaf6d1c6c6f0a70eb7d655b 789e09c2c59589392a122d7de0d22c17cad48131 refs/heads/master\\n\""}1844gitlab # [ 248.176153] gitaly[895]: time="2026-09-18T22:14:00.914Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:10.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:00.904" grpc.time_ms=10.052 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845gitlab # [ 248.188748] gitaly[895]: time="2026-09-18T22:14:00.926Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-18T22:14:10.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:14:00.918" grpc.time_ms=7.845 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1846gitlab # [ 248.204326] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:00.942Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"c52f4f844b2b754d8dcb4929","created_at":"2026-09-18T22:14:00.940Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-68762b4093714936","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"bb2f4e670f1764ab798c0856","job_size_bytes":46,"pid":2539,"job_status":"deduplicated","message":"ProjectCacheWorker JID-c52f4f844b2b754d8dcb4929: deduplicated: until executing","deduplication.type":"until executing"}1847gitlab # [ 248.242281] gitaly[895]: time="2026-09-18T22:14:00.979Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-18T22:14:10.070" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:00.970" grpc.time_ms=8.135 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1848gitlab # [ 248.257118] gitaly[895]: time="2026-09-18T22:14:00.995Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-18T22:14:30.484" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:00.985" grpc.time_ms=9.95 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1849gitlab # [ 248.276926] gitaly[895]: time="2026-09-18T22:14:01.015Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-18T22:14:11.008" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-18T22:14:01.008" grpc.time_ms=5.895 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1850gitlab # [ 248.289854] gitaly[895]: time="2026-09-18T22:14:01.027Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:11.022" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:01.023" grpc.time_ms=4.843 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1851gitlab # [ 248.348378] gitaly[895]: time="2026-09-18T22:14:01.086Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-18T22:14:11.085" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:01.085" grpc.time_ms=0.703 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1852gitlab # [ 248.374569] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:01.112Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","6057835189e4eae24eaf6d1c6c6f0a70eb7d655b 789e09c2c59589392a122d7de0d22c17cad48131 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"aae87a137938470b5916d494","created_at":"2026-09-18T22:13:58.491Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-58adf9f3ce064c98","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:13:58.493Z","job_size_bytes":185,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-aae87a137938470b5916d494: done: 0.269615 sec","job_status":"done","queue_duration_s":2.349418,"scheduling_latency_s":2.349461,"gitaly_calls":7,"gitaly_duration_s":0.090693,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.02241,"redis_read_bytes":501,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.002995,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000744,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003712,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.009331,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.005628,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.123974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000945651,"exclusive_lock_hold_duration_s":0.024248334,"duration_s":0.269615,"completed_at":"2026-09-18T22:14:01.112Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018039,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1853gitlab # [ 248.419283] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:01.157Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"bb2f4e670f1764ab798c0856","created_at":"2026-09-18T22:13:59.135Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.150Z","job_size_bytes":46,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-bb2f4e670f1764ab798c0856: start","job_status":"start","queue_duration_s":2.006825,"scheduling_latency_s":2.006854}1854gitlab # [ 248.485022] gitaly[895]: time="2026-09-18T22:14:01.223Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-19T04:14:01.220" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:14:01.220" grpc.time_ms=2.154 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1855gitlab # [ 248.506142] gitaly[895]: time="2026-09-18T22:14:01.244Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-18T22:14:31.034" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:01.234" grpc.time_ms=9.21 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1856gitlab # [ 248.526802] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:01.264Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"bb2f4e670f1764ab798c0856","created_at":"2026-09-18T22:13:59.135Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.150Z","job_size_bytes":46,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-bb2f4e670f1764ab798c0856: done: 0.10762 sec","job_status":"done","queue_duration_s":2.006825,"scheduling_latency_s":2.006854,"gitaly_calls":2,"gitaly_duration_s":0.015881,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007343000000000001,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001427,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.00051,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001648,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003001,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000294451,"duration_s":0.10762,"completed_at":"2026-09-18T22:14:01.264Z","load_balancing_strategy":"primary","db_duration_s":0.013192,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1857gitlab # [ 248.530701] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:01.265Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8f7289f73871ffa9129a7a66","created_at":"2026-09-18T22:13:59.163Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.166Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8f7289f73871ffa9129a7a66: start","job_status":"start","queue_duration_s":2.099028,"scheduling_latency_s":2.099043}1858gitlab # [ 248.622442] gitaly[895]: time="2026-09-18T22:14:01.360Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS diskcache=e9e0b815-3d6c-41bb-a48a-87da46a46679 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-19T04:14:01.335" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:01.336" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1859gitlab # [ 248.622854] gitaly[895]: time="2026-09-18T22:14:01.360Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1277 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-19T04:14:01.335" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:01.336" grpc.time_ms=24.039 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1860gitlab # [ 248.670792] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:01.408Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8f7289f73871ffa9129a7a66","created_at":"2026-09-18T22:13:59.163Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.166Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8f7289f73871ffa9129a7a66: done: 0.142647 sec","job_status":"done","queue_duration_s":2.099028,"scheduling_latency_s":2.099043,"gitaly_calls":1,"gitaly_duration_s":0.047706,"redis_calls":15,"redis_duration_s":0.00355,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000676,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000614,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.072386016,"duration_s":0.142647,"completed_at":"2026-09-18T22:14:01.408Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.062167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1861gitlab # [ 248.672942] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:01.409Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"bd15b63f25d3867170582db5","created_at":"2026-09-18T22:13:59.297Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.299Z","job_size_bytes":23,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-bd15b63f25d3867170582db5: start","job_status":"start","queue_duration_s":2.110334,"scheduling_latency_s":2.110349}1862gitlab # [ 248.713186] gitaly[895]: time="2026-09-18T22:14:01.437Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-18T22:14:31.227" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:01.428" grpc.time_ms=8.9 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1863gitlab # [ 248.745639] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:01.483Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"bd15b63f25d3867170582db5","created_at":"2026-09-18T22:13:59.297Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.299Z","job_size_bytes":23,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-bd15b63f25d3867170582db5: done: 0.073528 sec","job_status":"done","queue_duration_s":2.110334,"scheduling_latency_s":2.110349,"gitaly_calls":1,"gitaly_duration_s":0.048974,"redis_calls":16,"redis_duration_s":0.002969,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000269,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.00142,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.073528,"completed_at":"2026-09-18T22:14:01.483Z","load_balancing_strategy":"primary","db_duration_s":0.00202,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1864gitlab # [ 248.747071] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:01.484Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"91300e23d2f1c704d878cf26","created_at":"2026-09-18T22:13:59.884Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:74f719a7e730e0ef8b15e4107636860a840703c56d725e832d7a840be60fd3aa","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.887Z","job_size_bytes":467,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-91300e23d2f1c704d878cf26: start","job_status":"start","queue_duration_s":1.597165,"scheduling_latency_s":1.597186}1865gitlab # [ 248.792339] gitaly[895]: time="2026-09-18T22:14:01.505Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:31.304" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:01.504" grpc.time_ms=0.595 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866gitlab # [ 248.793167] gitaly[895]: time="2026-09-18T22:14:01.516Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-18T22:14:11.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:01.507" grpc.time_ms=8.783 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 248.881446] gitaly[895]: time="2026-09-18T22:14:01.619Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-18T22:14:31.417" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:01.618" grpc.time_ms=0.999 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868gitlab # [ 248.903452] gitaly[895]: time="2026-09-18T22:14:01.625Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=63 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-18T22:14:31.421" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:01.622" grpc.time_ms=2.883 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869gitlab # [ 248.970369] gitaly[895]: time="2026-09-18T22:14:01.708Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:11.202" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:01.702" grpc.time_ms=5.554 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1870gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131'1871gitlab # [ 249.089638] nixbot[2660]: INFO: ::ffff:127.0.0.1:42270 - "GET /api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 200 OK1872gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131', in 0.05 seconds)1873gitlab # [ 249.388445] gitaly[895]: time="2026-09-18T22:14:02.126Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=969 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-18T22:14:32.018" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:02.118" grpc.time_ms=8.177 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874gitlab # [ 249.416675] gitaly[895]: time="2026-09-18T22:14:02.154Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V99AWE1NSAGFHB2NW0TZMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-18T22:14:32.040" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:02.141" grpc.time_ms=13.482 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1875gitlab # [ 249.438205] sidekiq-cluster[2539]: {"severity":"WARN","time":"2026-09-18T22:14:02.175Z","class":"CreatePipelineWorker","correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"6057835189e4eae24eaf6d1c6c6f0a70eb7d655b","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"6057835189e4eae24eaf6d1c6c6f0a70eb7d655b","ref":"refs/heads/master"},"retry":0}1876gitlab # [ 249.443090] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.180Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"91300e23d2f1c704d878cf26","created_at":"2026-09-18T22:13:59.884Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:74f719a7e730e0ef8b15e4107636860a840703c56d725e832d7a840be60fd3aa","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.887Z","job_size_bytes":467,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-91300e23d2f1c704d878cf26: done: 0.696123 sec","job_status":"done","queue_duration_s":1.597165,"scheduling_latency_s":1.597186,"gitaly_calls":7,"gitaly_duration_s":0.065378,"redis_calls":51,"redis_duration_s":0.008092,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.000506,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003348,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001984,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.577174,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000360381,"duration_s":0.696123,"completed_at":"2026-09-18T22:14:02.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040199,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1877gitlab # [ 249.449240] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.182Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"1ee1ebc0-74cd-4abc-b511-c4af9aff1db3\"}"],"class":"WebHookWorker","jid":"318741e751c9a3d1984730a8","created_at":"2026-09-18T22:13:59.968Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8595f15f717a0c1ac031e5c50738d763b23f01c7566cbfb18250736981f0bf02","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.972Z","job_size_bytes":1751,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-318741e751c9a3d1984730a8: start","job_status":"start","queue_duration_s":2.209413,"scheduling_latency_s":2.209428}1878gitlab # [ 249.583648] nixbot[2660]: INFO: ::ffff:127.0.0.1:42286 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1879gitlab # [ 249.607871] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.344Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"1ee1ebc0-74cd-4abc-b511-c4af9aff1db3\"}"],"class":"WebHookWorker","jid":"318741e751c9a3d1984730a8","created_at":"2026-09-18T22:13:59.968Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-783401806e994162","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8595f15f717a0c1ac031e5c50738d763b23f01c7566cbfb18250736981f0bf02","size_limiter":"validated","enqueued_at":"2026-09-18T22:13:59.972Z","job_size_bytes":1751,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-318741e751c9a3d1984730a8: done: 0.16257 sec","job_status":"done","queue_duration_s":2.209413,"scheduling_latency_s":2.209428,"redis_calls":19,"redis_duration_s":0.003988,"redis_read_bytes":32,"redis_write_bytes":5366,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001113,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000325,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001722,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.03571599200000719,"cpu_s":0.07051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000446146,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.16257,"completed_at":"2026-09-18T22:14:02.344Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1880gitlab # [ 249.612257] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.348Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3485285c1a5269e556abc398","created_at":"2026-09-18T22:14:00.310Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.313Z","job_size_bytes":23,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3485285c1a5269e556abc398: start","job_status":"start","queue_duration_s":2.034799,"scheduling_latency_s":2.034835}1881gitlab # [ 249.687316] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.424Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3485285c1a5269e556abc398","created_at":"2026-09-18T22:14:00.310Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.313Z","job_size_bytes":23,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3485285c1a5269e556abc398: done: 0.075577 sec","job_status":"done","queue_duration_s":2.034799,"scheduling_latency_s":2.034835,"redis_calls":10,"redis_duration_s":0.002151,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000351,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001369,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030367,"duration_s":0.075577,"completed_at":"2026-09-18T22:14:02.424Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0118,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1882gitlab # [ 249.688363] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.425Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"32b65de2983b300c8fdf5c7e","created_at":"2026-09-18T22:14:00.549Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.557Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-32b65de2983b300c8fdf5c7e: start","job_status":"start","queue_duration_s":1.868255,"scheduling_latency_s":1.868271}1883gitlab # [ 249.734800] gitaly[895]: time="2026-09-18T22:14:02.472Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AWD59PAPGEW4VXJ3NKXA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:02.459" grpc.time_ms=12.946 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1884gitlab # [ 249.736449] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 661885gitlab # [ 249.737305] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.474Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"32b65de2983b300c8fdf5c7e","created_at":"2026-09-18T22:14:00.549Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.557Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-32b65de2983b300c8fdf5c7e: done: 0.048518 sec","job_status":"done","queue_duration_s":1.868255,"scheduling_latency_s":1.868271,"redis_calls":10,"redis_duration_s":0.005214,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001325,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040508,"duration_s":0.048518,"completed_at":"2026-09-18T22:14:02.474Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007153,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1886gitlab # [ 249.781696] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.476Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"576dbbdb3852b7ccfab469fd","created_at":"2026-09-18T22:14:00.575Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.582Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-576dbbdb3852b7ccfab469fd: start","job_status":"start","queue_duration_s":1.893454,"scheduling_latency_s":1.893469}1887gitlab # [ 249.782516] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.501Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"576dbbdb3852b7ccfab469fd","created_at":"2026-09-18T22:14:00.575Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.582Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-576dbbdb3852b7ccfab469fd: done: 0.024637 sec","job_status":"done","queue_duration_s":1.893454,"scheduling_latency_s":1.893469,"redis_calls":10,"redis_duration_s":0.0013180000000000002,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210641,"duration_s":0.024637,"completed_at":"2026-09-18T22:14:02.500Z","load_balancing_strategy":"primary","db_duration_s":0.004856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1888gitlab # [ 249.811550] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.502Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"83af36c3034bc4230f9256bc","created_at":"2026-09-18T22:14:00.603Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.609Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-83af36c3034bc4230f9256bc: start","job_status":"start","queue_duration_s":1.892785,"scheduling_latency_s":1.892823}1889gitlab # [ 249.825579] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.527Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"83af36c3034bc4230f9256bc","created_at":"2026-09-18T22:14:00.603Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.609Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-83af36c3034bc4230f9256bc: done: 0.025447 sec","job_status":"done","queue_duration_s":1.892785,"scheduling_latency_s":1.892823,"redis_calls":13,"redis_duration_s":0.001897,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000349,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000378,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000221536,"duration_s":0.025447,"completed_at":"2026-09-18T22:14:02.527Z","load_balancing_strategy":"primary","db_duration_s":0.004397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1890gitlab # [ 249.892246] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.528Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"14527bc7db798f499df8f30e","created_at":"2026-09-18T22:14:00.610Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.615Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-14527bc7db798f499df8f30e: start","job_status":"start","queue_duration_s":1.912847,"scheduling_latency_s":1.91286}1891gitlab # [ 249.892979] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.548Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"14527bc7db798f499df8f30e","created_at":"2026-09-18T22:14:00.610Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.615Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-14527bc7db798f499df8f30e: done: 0.020285 sec","job_status":"done","queue_duration_s":1.912847,"scheduling_latency_s":1.91286,"redis_calls":10,"redis_duration_s":0.001214,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000244,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000236,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00019332,"duration_s":0.020285,"completed_at":"2026-09-18T22:14:02.548Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1892gitlab # [ 249.944525] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.549Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f101eed3a0f4c72a26cf1df7","created_at":"2026-09-18T22:14:00.616Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.622Z","job_size_bytes":19,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f101eed3a0f4c72a26cf1df7: start","job_status":"start","queue_duration_s":1.927386,"scheduling_latency_s":1.927398}1893gitlab # [ 249.945598] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.568Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f101eed3a0f4c72a26cf1df7","created_at":"2026-09-18T22:14:00.616Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.622Z","job_size_bytes":19,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f101eed3a0f4c72a26cf1df7: done: 0.018295 sec","job_status":"done","queue_duration_s":1.927386,"scheduling_latency_s":1.927398,"redis_calls":11,"redis_duration_s":0.001346,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000190806,"duration_s":0.018295,"completed_at":"2026-09-18T22:14:02.568Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002364,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1894gitlab # [ 249.977721] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.568Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7196bed741e5f3a56252fb7f","created_at":"2026-09-18T22:14:00.625Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.629Z","job_size_bytes":66,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7196bed741e5f3a56252fb7f: start","job_status":"start","queue_duration_s":1.939381,"scheduling_latency_s":1.939394}1895gitlab # [ 250.033947] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.588Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7196bed741e5f3a56252fb7f","created_at":"2026-09-18T22:14:00.625Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.629Z","job_size_bytes":66,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7196bed741e5f3a56252fb7f: done: 0.019471 sec","job_status":"done","queue_duration_s":1.939381,"scheduling_latency_s":1.939394,"redis_calls":10,"redis_duration_s":0.001276,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000223492,"duration_s":0.019471,"completed_at":"2026-09-18T22:14:02.588Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002848,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1896gitlab # [ 250.036869] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.589Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"be212d95280df657897ff168","created_at":"2026-09-18T22:14:00.630Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.633Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-be212d95280df657897ff168: start","job_status":"start","queue_duration_s":1.956123,"scheduling_latency_s":1.956136}1897gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131'1898gitlab # [ 250.076728] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.613Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"be212d95280df657897ff168","created_at":"2026-09-18T22:14:00.630Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.633Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-be212d95280df657897ff168: done: 0.023805 sec","job_status":"done","queue_duration_s":1.956123,"scheduling_latency_s":1.956136,"redis_calls":10,"redis_duration_s":0.001441,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000425,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000221257,"duration_s":0.023805,"completed_at":"2026-09-18T22:14:02.613Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1899gitlab # [ 250.116285] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.614Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"aac4b001c83a3c00206bb811","created_at":"2026-09-18T22:14:00.639Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.644Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-aac4b001c83a3c00206bb811: start","job_status":"start","queue_duration_s":1.969301,"scheduling_latency_s":1.969314}1900gitlab # [ 250.117332] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.637Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"aac4b001c83a3c00206bb811","created_at":"2026-09-18T22:14:00.639Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.644Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-aac4b001c83a3c00206bb811: done: 0.023514 sec","job_status":"done","queue_duration_s":1.969301,"scheduling_latency_s":1.969314,"redis_calls":10,"redis_duration_s":0.00133,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000238,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214552,"duration_s":0.023514,"completed_at":"2026-09-18T22:14:02.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1901gitlab # [ 250.164365] gitaly[895]: time="2026-09-18T22:14:02.823Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR diskcache=a4e0ab6b-5e72-4e58-998d-7e7fca4b350a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-19T04:14:02.752" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:02.752" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1902gitlab # [ 250.165142] gitaly[895]: time="2026-09-18T22:14:02.823Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-19T04:14:02.752" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:02.752" grpc.time_ms=70.796 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1903gitlab # [ 250.165818] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.638Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"fe5049c56fd9f5bbd6a947a0","created_at":"2026-09-18T22:14:00.645Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.650Z","job_size_bytes":6,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-fe5049c56fd9f5bbd6a947a0: start","job_status":"start","queue_duration_s":1.987937,"scheduling_latency_s":1.98795}1904gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131', in 0.13 seconds)1905gitlab # [ 250.178641] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.652Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"fe5049c56fd9f5bbd6a947a0","created_at":"2026-09-18T22:14:00.645Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.650Z","job_size_bytes":6,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-fe5049c56fd9f5bbd6a947a0: done: 0.01401 sec","job_status":"done","queue_duration_s":1.987937,"scheduling_latency_s":1.98795,"redis_calls":10,"redis_duration_s":0.001229,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000241,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000247,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197511,"duration_s":0.01401,"completed_at":"2026-09-18T22:14:02.652Z","load_balancing_strategy":"primary","db_duration_s":0.001346,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1906gitlab # [ 250.223971] gitaly[895]: time="2026-09-18T22:14:02.922Z" level=info msg="finished unary call with code OK" catfile.duration_ms=12 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=12 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:32.807" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:02.909" grpc.time_ms=13.568 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907gitlab # [ 250.229416] nixbot[2660]: INFO: ::ffff:127.0.0.1:42298 - "GET /api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 200 OK1908gitlab # [ 250.230820] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.653Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"50f2328e2395ea246f45aee4","created_at":"2026-09-18T22:14:00.651Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.657Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-50f2328e2395ea246f45aee4: start","job_status":"start","queue_duration_s":1.996268,"scheduling_latency_s":1.996281}1909gitlab # [ 250.276238] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.675Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"50f2328e2395ea246f45aee4","created_at":"2026-09-18T22:14:00.651Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.657Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-50f2328e2395ea246f45aee4: done: 0.021928 sec","job_status":"done","queue_duration_s":1.996268,"scheduling_latency_s":1.996281,"redis_calls":10,"redis_duration_s":0.001275,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000234,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226845,"duration_s":0.021928,"completed_at":"2026-09-18T22:14:02.675Z","load_balancing_strategy":"primary","db_duration_s":0.004158,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1910gitlab # [ 250.332907] gitaly[895]: time="2026-09-18T22:14:02.934Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-18T22:14:12.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:02.927" grpc.time_ms=6.866 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911gitlab # [ 250.333797] gitlab-workhorse[828]: time="2026-09-18T22:14:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671912gitlab # [ 250.334417] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 641913gitlab # [ 250.335105] gitlab-workhorse[828]: time="2026-09-18T22:14:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3361914gitlab # [ 250.335569] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 591915gitlab # [ 250.336122] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.676Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f6e9cb63b8dc6a4b4b629599","created_at":"2026-09-18T22:14:00.664Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.671Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f6e9cb63b8dc6a4b4b629599: start","job_status":"start","queue_duration_s":2.004918,"scheduling_latency_s":2.004956}1916gitlab # [ 250.336795] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.717Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f6e9cb63b8dc6a4b4b629599","created_at":"2026-09-18T22:14:00.664Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.671Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f6e9cb63b8dc6a4b4b629599: done: 0.041 sec","job_status":"done","queue_duration_s":2.004918,"scheduling_latency_s":2.004956,"redis_calls":14,"redis_duration_s":0.002128,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00024,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000774,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205054,"duration_s":0.041,"completed_at":"2026-09-18T22:14:02.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008454,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1917gitlab # [ 250.380844] gitaly[895]: time="2026-09-18T22:14:02.971Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-18T22:14:32.869" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:02.970" grpc.time_ms=0.991 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918gitlab # [ 250.381813] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.718Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"b4b6c39e01afdfe82009bb0f","created_at":"2026-09-18T22:14:00.765Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.783Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b4b6c39e01afdfe82009bb0f: start","job_status":"start","queue_duration_s":1.935347,"scheduling_latency_s":1.935362}1919gitlab # [ 250.422523] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.736Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"b4b6c39e01afdfe82009bb0f","created_at":"2026-09-18T22:14:00.765Z","trace_propagation_headers":{"sentry-trace":"965e870b88ee428dbe9f7260a6ef4be9-faa5e5f5d242485c","baggage":"sentry-trace_id=965e870b88ee428dbe9f7260a6ef4be9,sentry-release=Unknown"},"correlation_id":"01M2V99F9H7ZQXSYMHK58302RP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.783Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b4b6c39e01afdfe82009bb0f: done: 0.018432 sec","job_status":"done","queue_duration_s":1.935347,"scheduling_latency_s":1.935362,"redis_calls":8,"redis_duration_s":0.001371,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001083,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018432,"completed_at":"2026-09-18T22:14:02.736Z","load_balancing_strategy":"primary","db_duration_s":0.002766,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1920gitlab # [ 250.430704] gitaly[895]: time="2026-09-18T22:14:02.977Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=298 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-18T22:14:32.873" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:02.973" grpc.time_ms=3.614 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921gitlab # [ 250.472055] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.738Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"646445d2db5b1632d93247e0","created_at":"2026-09-18T22:14:00.961Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-68762b4093714936","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8f7289f73871ffa9129a7a66","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.965Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-646445d2db5b1632d93247e0: start","job_status":"start","queue_duration_s":1.772784,"scheduling_latency_s":1.7728}1922gitlab # [ 250.472730] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.841Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"646445d2db5b1632d93247e0","created_at":"2026-09-18T22:14:00.961Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-68762b4093714936","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8f7289f73871ffa9129a7a66","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:00.965Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-646445d2db5b1632d93247e0: done: 0.102939 sec","job_status":"done","queue_duration_s":1.772784,"scheduling_latency_s":1.7728,"gitaly_calls":1,"gitaly_duration_s":0.073265,"redis_calls":12,"redis_duration_s":0.003238,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000588,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001393,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08709403,"duration_s":0.102939,"completed_at":"2026-09-18T22:14:02.841Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1923gitlab # [ 250.513876] gitaly[895]: time="2026-09-18T22:14:02.987Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AWXBCYW94FCGNTEBKEXJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:02.979" pid=895 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1924gitlab # [ 250.514701] nixbot[2660]: {"ts": "2026-09-18T22:14:03.173Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1925gitlab # [ 250.515149] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:02.842Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"315fc03b3edb966f374b4f3b","created_at":"2026-09-18T22:14:01.066Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-68762b4093714936","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a5238f96b87c7916c42e83f20529d92fd3a0214c98e25effb92e3294ab27bd08","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:01.067Z","job_size_bytes":467,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-315fc03b3edb966f374b4f3b: start","job_status":"start","queue_duration_s":1.774989,"scheduling_latency_s":1.775009}1926gitlab # [ 250.557580] gitaly[895]: time="2026-09-18T22:14:02.987Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AWXBCYW94FCGNTEBKEXJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:02.979" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1927gitlab # [ 250.558461] gitaly[895]: time="2026-09-18T22:14:02.987Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AWXBCYW94FCGNTEBKEXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:02.979" grpc.time_ms=8.107 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1928gitlab # [ 250.559126] gitaly[895]: time="2026-09-18T22:14:03.010Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:13.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:03.005" grpc.time_ms=5.452 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 250.562019] gitaly[895]: time="2026-09-18T22:14:03.048Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AWZEE5RXR8KE6RKN92CE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.035" pid=895 remote_ip="2001:db8:1::1" request_sha=c66427a96997d92ea19be2c2a80aa1085d78f51d response_bytes=336 span.kind=server system=grpc1930gitlab # [ 250.604787] gitaly[895]: time="2026-09-18T22:14:03.048Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1189 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AWZEE5RXR8KE6RKN92CE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.035" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1931gitlab # [ 250.605682] gitaly[895]: time="2026-09-18T22:14:03.049Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1189 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AWZEE5RXR8KE6RKN92CE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.035" grpc.time_ms=13.847 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1932gitlab # [ 250.660136] gitaly[895]: time="2026-09-18T22:14:03.398Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:33.397" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:03.397" grpc.time_ms=0.538 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933gitlab # [ 250.688687] gitaly[895]: time="2026-09-18T22:14:03.426Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=965 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-18T22:14:33.317" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:03.417" grpc.time_ms=9.062 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1934gitlab # [ 250.701994] gitaly[895]: time="2026-09-18T22:14:03.438Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AR43XCZ1HAN176AP1RSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-18T22:14:33.329" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:03.430" grpc.time_ms=8.253 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935gitlab # [ 250.716408] sidekiq-cluster[2539]: {"severity":"WARN","time":"2026-09-18T22:14:03.454Z","class":"CreatePipelineWorker","correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"6057835189e4eae24eaf6d1c6c6f0a70eb7d655b","after":"789e09c2c59589392a122d7de0d22c17cad48131","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"789e09c2c59589392a122d7de0d22c17cad48131","ref":"refs/heads/master"},"retry":0}1936gitlab # [ 250.723304] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:03.460Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"315fc03b3edb966f374b4f3b","created_at":"2026-09-18T22:14:01.066Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-68762b4093714936","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a5238f96b87c7916c42e83f20529d92fd3a0214c98e25effb92e3294ab27bd08","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:01.067Z","job_size_bytes":467,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-315fc03b3edb966f374b4f3b: done: 0.61824 sec","job_status":"done","queue_duration_s":1.774989,"scheduling_latency_s":1.775009,"gitaly_calls":7,"gitaly_duration_s":0.066295,"redis_calls":33,"redis_duration_s":0.044752,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.003793,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001663,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.038433,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.449279,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.61824,"completed_at":"2026-09-18T22:14:03.460Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022009,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1937gitlab # [ 250.726166] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:03.464Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"0342de4f-af09-4be8-ac17-36f15e72e5ba\"}"],"class":"WebHookWorker","jid":"2f089d5c83c168e86173a71f","created_at":"2026-09-18T22:14:01.076Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-68762b4093714936","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6c4b56866294e19e63c0fdb58584e87ef4331134b5c92112a172a1e73eace50a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:01.077Z","job_size_bytes":1749,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2f089d5c83c168e86173a71f: start","job_status":"start","queue_duration_s":2.386479,"scheduling_latency_s":2.3865}1938gitlab # [ 250.766268] gitaly[895]: time="2026-09-18T22:14:03.466Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:13.263" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:03.454" grpc.time_ms=12.289 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1939gitlab # [ 250.766928] nixbot[2660]: INFO: ::ffff:127.0.0.1:42308 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1940gitlab # [ 250.773526] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:03.510Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"0342de4f-af09-4be8-ac17-36f15e72e5ba\"}"],"class":"WebHookWorker","jid":"2f089d5c83c168e86173a71f","created_at":"2026-09-18T22:14:01.076Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-68762b4093714936","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6c4b56866294e19e63c0fdb58584e87ef4331134b5c92112a172a1e73eace50a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:01.077Z","job_size_bytes":1749,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2f089d5c83c168e86173a71f: done: 0.047017 sec","job_status":"done","queue_duration_s":2.386479,"scheduling_latency_s":2.3865,"redis_calls":14,"redis_duration_s":0.002238,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000414,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001294,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.018093639999989364,"cpu_s":0.019944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.047017,"completed_at":"2026-09-18T22:14:03.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1941gitlab # [ 250.797367] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:03.516Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ae3e6482ae666ad0a418c10f","created_at":"2026-09-18T22:14:01.252Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-d194608d359d4b69","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:01.256Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ae3e6482ae666ad0a418c10f: start","job_status":"start","queue_duration_s":2.260315,"scheduling_latency_s":2.260337}1942gitlab # [ 250.844700] nixbot[2660]: {"ts": "2026-09-18T22:14:03.519Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b \"HTTP/1.1 201 Created\""}1943gitlab # [ 250.845763] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 2981944gitlab # [ 250.846447] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:03.558Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ae3e6482ae666ad0a418c10f","created_at":"2026-09-18T22:14:01.252Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-d194608d359d4b69","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:01.256Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ae3e6482ae666ad0a418c10f: done: 0.042021 sec","job_status":"done","queue_duration_s":2.260315,"scheduling_latency_s":2.260337,"redis_calls":19,"redis_duration_s":0.009374,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001396,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000342,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005992,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001644,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007335011,"duration_s":0.042021,"completed_at":"2026-09-18T22:14:03.558Z","load_balancing_strategy":"primary","db_duration_s":0.004749,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1945gitlab # [ 250.858953] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:03.562Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","318741e751c9a3d1984730a8"],"class":"WebHooks::LogExecutionWorker","jid":"8ed15e8622b002b6141b38b0","created_at":"2026-09-18T22:14:02.326Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-6f5c4d36caba4ad6","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f8bf6d0b0a51b6b55a28e73574f86b7958b9e651684fe363ac3c917019ee5799","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:02.334Z","job_size_bytes":2593,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8ed15e8622b002b6141b38b0: start","job_status":"start","queue_duration_s":1.227627,"scheduling_latency_s":1.227649}1946gitlab # [ 250.906125] gitaly[895]: time="2026-09-18T22:14:03.566Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXG2N8ZZWG4TGSSAS0JM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:33.564" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:03.565" grpc.time_ms=1.396 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1947gitlab # [ 250.906802] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 551948gitlab # [ 250.907547] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:03.609Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","318741e751c9a3d1984730a8"],"class":"WebHooks::LogExecutionWorker","jid":"8ed15e8622b002b6141b38b0","created_at":"2026-09-18T22:14:02.326Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-6f5c4d36caba4ad6","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f8bf6d0b0a51b6b55a28e73574f86b7958b9e651684fe363ac3c917019ee5799","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:02.334Z","job_size_bytes":2593,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8ed15e8622b002b6141b38b0: done: 0.04754 sec","job_status":"done","queue_duration_s":1.227627,"scheduling_latency_s":1.227649,"redis_calls":10,"redis_duration_s":0.004528,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001987,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002241,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000253943,"duration_s":0.04754,"completed_at":"2026-09-18T22:14:03.609Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009955,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1949gitlab # [ 250.925762] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:03.610Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3c1902db7bb42123a284bbad","created_at":"2026-09-18T22:14:03.502Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-e4ff2cee412e4e15","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.505Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3c1902db7bb42123a284bbad: start","job_status":"start","queue_duration_s":0.105717,"scheduling_latency_s":0.105733}1950gitlab # [ 250.982683] gitaly[895]: time="2026-09-18T22:14:03.608Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AXH0KKJT2Y9617DNSQX4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.600" grpc.time_ms=7.248 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1951gitlab # [ 250.983633] gitaly[895]: time="2026-09-18T22:14:03.663Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXJSFD5GZX50F8J76DWP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.653" pid=895 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1952gitlab # [ 250.984234] gitlab-workhorse[828]: time="2026-09-18T22:14:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671953gitlab # [ 250.984753] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 541954gitlab # [ 250.988124] gitaly[895]: time="2026-09-18T22:14:03.663Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXJSFD5GZX50F8J76DWP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.653" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1955gitlab # [ 250.988761] gitaly[895]: time="2026-09-18T22:14:03.663Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXJSFD5GZX50F8J76DWP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.653" grpc.time_ms=10.098 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1956gitlab # [ 250.997513] gitaly[895]: time="2026-09-18T22:14:03.690Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=92 command.spawn_token_fork_ms=89 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AXG2N8ZZWG4TGSSAS0JM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:13.267" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:03.597" grpc.time_ms=93.215 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957gitlab # [ 251.009070] gitaly[895]: time="2026-09-18T22:14:03.746Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:03.746" grpc.time_ms=0.276 pid=895 span.kind=server system=grpc1958gitlab # [ 251.037077] gitaly[895]: time="2026-09-18T22:14:03.775Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:03.774" grpc.time_ms=0.257 pid=895 span.kind=server system=grpc1959gitlab # [ 251.038508] gitaly[895]: time="2026-09-18T22:14:03.776Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AXND2P83YYE11FQDFSHG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.765" grpc.time_ms=10.766 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1960gitlab # [ 251.040762] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 831961gitlab # [ 251.051710] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2661962gitlab # [ 251.054318] nixbot[2660]: {"ts": "2026-09-18T22:14:03.791Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b \"HTTP/1.1 201 Created\""}1963gitlab # [ 251.058795] gitaly[895]: time="2026-09-18T22:14:03.796Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:03.796" grpc.time_ms=0.498 pid=895 span.kind=server system=grpc1964gitlab # [ 251.067098] gitaly[895]: time="2026-09-18T22:14:03.805Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 diskcache=5057caaa-374c-46d2-8f63-7dc04a2828c7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-18T22:14:33.623" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:03.724" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1965gitlab # [ 251.067922] gitaly[895]: time="2026-09-18T22:14:03.805Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7183 command.oublock=0 command.real_time_ms=74 command.spawn_token_fork_ms=3 command.system_time_ms=39 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-18T22:14:33.623" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:03.724" grpc.time_ms=80.941 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966gitlab # [ 251.121664] gitaly[895]: time="2026-09-18T22:14:03.859Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-18T22:14:33.758" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:03.858" grpc.time_ms=0.698 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 251.145174] gitaly[895]: time="2026-09-18T22:14:03.883Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXR2G9N8AACXT7B472PG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.873" pid=895 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1968gitlab # [ 251.145599] gitaly[895]: time="2026-09-18T22:14:03.883Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXR2G9N8AACXT7B472PG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.873" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1969gitlab # [ 251.147313] gitaly[895]: time="2026-09-18T22:14:03.885Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXR2G9N8AACXT7B472PG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.873" grpc.time_ms=10.611 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1970gitlab # [ 251.148026] gitlab-workhorse[828]: time="2026-09-18T22:14:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671971gitlab # [ 251.148450] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1061972gitlab # [ 251.187088] gitaly[895]: time="2026-09-18T22:14:03.925Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXRK408T89J5Y3JHKW70 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:33.024" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:03.924" grpc.time_ms=0.434 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1973gitlab # [ 251.206102] gitaly[895]: time="2026-09-18T22:14:03.943Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:33.842" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:03.942" grpc.time_ms=0.797 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131'1975gitlab # [ 251.226862] gitaly[895]: time="2026-09-18T22:14:03.963Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AXW90RP2Q80FFP6J04GQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:03.955" grpc.time_ms=7.027 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1976gitlab # [ 251.227109] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 511977gitlab # [ 251.263547] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.001Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"ef4164068528f411e41cc3e5","created_at":"2026-09-18T22:14:03.998Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"3c1902db7bb42123a284bbad","job_size_bytes":3,"pid":2539,"job_status":"deduplicated","message":"PipelineProcessWorker JID-ef4164068528f411e41cc3e5: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1978gitlab # [ 251.265966] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.004Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3c1902db7bb42123a284bbad","created_at":"2026-09-18T22:14:03.502Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-e4ff2cee412e4e15","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.505Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2539,"job_status":"rescheduled","message":"PipelineProcessWorker JID-3c1902db7bb42123a284bbad: rescheduled"}1979gitlab # [ 251.274886] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.012Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3c1902db7bb42123a284bbad","created_at":"2026-09-18T22:14:03.502Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-e4ff2cee412e4e15","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.505Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3c1902db7bb42123a284bbad: done: 0.401577 sec","job_status":"done","queue_duration_s":0.105717,"scheduling_latency_s":0.105733,"gitaly_calls":3,"gitaly_duration_s":0.089525,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.03579,"redis_read_bytes":228,"redis_write_bytes":18463,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000581,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00074,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00716,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12309,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.009754,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001646,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001934,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.013394,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.135,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":18,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.114,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121605,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.317073538,"duration_s":0.401577,"completed_at":"2026-09-18T22:14:04.012Z","load_balancing_strategy":"primary","db_duration_s":0.136574,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1980gitlab # [ 251.328855] gitaly[895]: time="2026-09-18T22:14:04.020Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AXRK408T89J5Y3JHKW70 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:14.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:04.014" grpc.time_ms=5.697 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1981gitlab # [ 251.329555] gitlab-workhorse[828]: time="2026-09-18T22:14:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161982gitlab # [ 251.330309] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:04 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 831983gitlab # [ 251.331648] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.017Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2f089d5c83c168e86173a71f"],"class":"WebHooks::LogExecutionWorker","jid":"9ea26b43d848f3d888a97f32","created_at":"2026-09-18T22:14:03.501Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-015d20bb99cf432b","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:53540483dc032a00559137040d1566585962db2f4426b009f6e19479217aae0f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.506Z","job_size_bytes":2592,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9ea26b43d848f3d888a97f32: start","job_status":"start","queue_duration_s":0.511447,"scheduling_latency_s":0.511471}1984gitlab # [ 251.332419] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.061Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2f089d5c83c168e86173a71f"],"class":"WebHooks::LogExecutionWorker","jid":"9ea26b43d848f3d888a97f32","created_at":"2026-09-18T22:14:03.501Z","trace_propagation_headers":{"sentry-trace":"7ed6f99da7c14c04a740d3b76362e18d-015d20bb99cf432b","baggage":"sentry-trace_id=7ed6f99da7c14c04a740d3b76362e18d,sentry-release=Unknown"},"correlation_id":"01M2V9AR43XCZ1HAN176AP1RSR","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:53540483dc032a00559137040d1566585962db2f4426b009f6e19479217aae0f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.506Z","job_size_bytes":2592,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9ea26b43d848f3d888a97f32: done: 0.04348 sec","job_status":"done","queue_duration_s":0.511447,"scheduling_latency_s":0.511471,"redis_calls":7,"redis_duration_s":0.001263,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.019,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.019,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04348,"completed_at":"2026-09-18T22:14:04.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1985gitlab # [ 251.376766] gitaly[895]: time="2026-09-18T22:14:04.049Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXXYCNBRKEVPBPF3MGEC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:04.037" pid=895 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc1986gitlab # [ 251.377595] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.062Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"6b23bf7df024437a9388285f","created_at":"2026-09-18T22:14:03.545Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-550d48329f5a4e68","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.549Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-6b23bf7df024437a9388285f: start","job_status":"start","queue_duration_s":0.512606,"scheduling_latency_s":0.512647}1987gitlab # [ 251.390928] gitaly[895]: time="2026-09-18T22:14:04.049Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXXYCNBRKEVPBPF3MGEC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:04.037" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1988gitlab # [ 251.430073] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 3331989gitlab # [ 251.430757] gitaly[895]: time="2026-09-18T22:14:04.049Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AXXYCNBRKEVPBPF3MGEC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:04.037" grpc.time_ms=12.593 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc1990gitlab # [ 251.431580] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.169Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"6b23bf7df024437a9388285f","created_at":"2026-09-18T22:14:03.545Z","trace_propagation_headers":{"sentry-trace":"0ad07af344c64a72bdcbccf00afa80bd-550d48329f5a4e68","baggage":"sentry-trace_id=0ad07af344c64a72bdcbccf00afa80bd,sentry-release=Unknown"},"correlation_id":"01M2V99AWE1NSAGFHB2NW0TZMS","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.549Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-6b23bf7df024437a9388285f: done: 0.106887 sec","job_status":"done","queue_duration_s":0.512606,"scheduling_latency_s":0.512647,"redis_calls":11,"redis_duration_s":0.008168,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000487,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00503,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002651,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.027,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.031,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208686,"duration_s":0.106887,"completed_at":"2026-09-18T22:14:04.169Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042057,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991gitlab # [ 251.472494] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.170Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8fd893c327db0ad9947419b4","created_at":"2026-09-18T22:14:03.715Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.718Z","job_size_bytes":23,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8fd893c327db0ad9947419b4: start","job_status":"start","queue_duration_s":0.452449,"scheduling_latency_s":0.452463}1992gitlab # [ 251.475807] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.197Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8fd893c327db0ad9947419b4","created_at":"2026-09-18T22:14:03.715Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.718Z","job_size_bytes":23,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8fd893c327db0ad9947419b4: done: 0.027314 sec","job_status":"done","queue_duration_s":0.452449,"scheduling_latency_s":0.452463,"redis_calls":7,"redis_duration_s":0.001014,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027314,"completed_at":"2026-09-18T22:14:04.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004766,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1993gitlab # [ 251.514916] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.198Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"560b8210b15ee62873cb553d","created_at":"2026-09-18T22:14:03.809Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.812Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-560b8210b15ee62873cb553d: start","job_status":"start","queue_duration_s":0.386706,"scheduling_latency_s":0.386723}1994gitlab # [ 251.524528] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.221Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"560b8210b15ee62873cb553d","created_at":"2026-09-18T22:14:03.809Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.812Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-560b8210b15ee62873cb553d: done: 0.02222 sec","job_status":"done","queue_duration_s":0.386706,"scheduling_latency_s":0.386723,"redis_calls":7,"redis_duration_s":0.001171,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000363,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02222,"completed_at":"2026-09-18T22:14:04.221Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1995gitlab # [ 251.568811] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.222Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d2f87d6faf01e65d4207791d","created_at":"2026-09-18T22:14:03.817Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.819Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d2f87d6faf01e65d4207791d: start","job_status":"start","queue_duration_s":0.402513,"scheduling_latency_s":0.402528}1996gitlab # [ 251.576450] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.252Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d2f87d6faf01e65d4207791d","created_at":"2026-09-18T22:14:03.817Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.819Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d2f87d6faf01e65d4207791d: done: 0.030756 sec","job_status":"done","queue_duration_s":0.402513,"scheduling_latency_s":0.402528,"redis_calls":7,"redis_duration_s":0.001031,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030756,"completed_at":"2026-09-18T22:14:04.252Z","load_balancing_strategy":"primary","db_duration_s":0.015592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1997gitlab # [ 251.625868] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.253Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"40f845e75dc5eb5f2aa5ff4c","created_at":"2026-09-18T22:14:03.827Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.829Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-40f845e75dc5eb5f2aa5ff4c: start","job_status":"start","queue_duration_s":0.423966,"scheduling_latency_s":0.423981}1998gitlab # [ 251.626828] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.273Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"40f845e75dc5eb5f2aa5ff4c","created_at":"2026-09-18T22:14:03.827Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.829Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-40f845e75dc5eb5f2aa5ff4c: done: 0.019825 sec","job_status":"done","queue_duration_s":0.423966,"scheduling_latency_s":0.423981,"redis_calls":10,"redis_duration_s":0.002092,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000708,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000379,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019825,"completed_at":"2026-09-18T22:14:04.273Z","load_balancing_strategy":"primary","db_duration_s":0.00312,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1999gitlab # [ 251.665758] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.274Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5ebe06c4ae6610b47a5f9f73","created_at":"2026-09-18T22:14:03.830Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.831Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5ebe06c4ae6610b47a5f9f73: start","job_status":"start","queue_duration_s":0.44282,"scheduling_latency_s":0.442835}2000gitlab # [ 251.705551] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.289Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5ebe06c4ae6610b47a5f9f73","created_at":"2026-09-18T22:14:03.830Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.831Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5ebe06c4ae6610b47a5f9f73: done: 0.0152 sec","job_status":"done","queue_duration_s":0.44282,"scheduling_latency_s":0.442835,"redis_calls":7,"redis_duration_s":0.000951,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0152,"completed_at":"2026-09-18T22:14:04.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002582,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2001gitlab # [ 251.717723] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.290Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"84977399d05106e5c99d9d5a","created_at":"2026-09-18T22:14:03.832Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.838Z","job_size_bytes":19,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-84977399d05106e5c99d9d5a: start","job_status":"start","queue_duration_s":0.451922,"scheduling_latency_s":0.451935}2002gitlab # [ 251.732700] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.304Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"84977399d05106e5c99d9d5a","created_at":"2026-09-18T22:14:03.832Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.838Z","job_size_bytes":19,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-84977399d05106e5c99d9d5a: done: 0.014117 sec","job_status":"done","queue_duration_s":0.451922,"scheduling_latency_s":0.451935,"redis_calls":8,"redis_duration_s":0.001037,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000229,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014117,"completed_at":"2026-09-18T22:14:04.304Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001989,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2003gitlab # [ 251.773674] gitaly[895]: time="2026-09-18T22:14:04.421Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:04.420" grpc.time_ms=0.359 pid=895 span.kind=server system=grpc2004gitlab # [ 251.812635] gitaly[895]: time="2026-09-18T22:14:04.439Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:04.438" grpc.time_ms=0.327 pid=895 span.kind=server system=grpc2005gitlab # [ 251.813513] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.305Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8a6c6a070ee583247d5445c7","created_at":"2026-09-18T22:14:03.842Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.843Z","job_size_bytes":66,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8a6c6a070ee583247d5445c7: start","job_status":"start","queue_duration_s":0.461733,"scheduling_latency_s":0.461746}2006gitlab # [ 251.814128] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.318Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8a6c6a070ee583247d5445c7","created_at":"2026-09-18T22:14:03.842Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.843Z","job_size_bytes":66,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8a6c6a070ee583247d5445c7: done: 0.013228 sec","job_status":"done","queue_duration_s":0.461733,"scheduling_latency_s":0.461746,"redis_calls":7,"redis_duration_s":0.0008860000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013228,"completed_at":"2026-09-18T22:14:04.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2007gitlab # [ 251.853867] gitaly[895]: time="2026-09-18T22:14:04.457Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:04.457" grpc.time_ms=0.257 pid=895 span.kind=server system=grpc2008gitlab # [ 251.854678] gitaly[895]: time="2026-09-18T22:14:04.465Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 diskcache=bbeed433-bbbd-4138-8e48-64a77a083fd1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-18T22:14:34.301" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:04.401" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2009gitlab # [ 251.857613] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.319Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"89157301c71b2e2ee0710f83","created_at":"2026-09-18T22:14:03.844Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.845Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-89157301c71b2e2ee0710f83: start","job_status":"start","queue_duration_s":0.474366,"scheduling_latency_s":0.47438}2010gitlab # [ 251.901778] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.336Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"89157301c71b2e2ee0710f83","created_at":"2026-09-18T22:14:03.844Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.845Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-89157301c71b2e2ee0710f83: done: 0.016687 sec","job_status":"done","queue_duration_s":0.474366,"scheduling_latency_s":0.47438,"redis_calls":7,"redis_duration_s":0.000908,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016687,"completed_at":"2026-09-18T22:14:04.336Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2011gitlab # [ 251.948146] gitaly[895]: time="2026-09-18T22:14:04.466Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=64 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7248 command.oublock=0 command.real_time_ms=57 command.spawn_token_fork_ms=0 command.system_time_ms=46 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-18T22:14:34.301" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:04.401" grpc.time_ms=64.396 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2012gitlab # [ 251.948771] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.337Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"bf0b5298f544c3b479126254","created_at":"2026-09-18T22:14:03.846Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.848Z","job_size_bytes":6,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-bf0b5298f544c3b479126254: start","job_status":"start","queue_duration_s":0.489183,"scheduling_latency_s":0.489197}2013gitlab # [ 251.949559] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.345Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"bf0b5298f544c3b479126254","created_at":"2026-09-18T22:14:03.846Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.848Z","job_size_bytes":6,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-bf0b5298f544c3b479126254: done: 0.008633 sec","job_status":"done","queue_duration_s":0.489183,"scheduling_latency_s":0.489197,"redis_calls":7,"redis_duration_s":0.000879,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008633,"completed_at":"2026-09-18T22:14:04.345Z","load_balancing_strategy":"primary","db_duration_s":0.000237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 251.998985] gitaly[895]: time="2026-09-18T22:14:04.567Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=62 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=62 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-18T22:14:34.403" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:04.503" grpc.time_ms=63.009 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2015gitlab # [ 251.999667] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.346Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"82d4c8407bbf40ea1cac2496","created_at":"2026-09-18T22:14:03.848Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.849Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-82d4c8407bbf40ea1cac2496: start","job_status":"start","queue_duration_s":0.496911,"scheduling_latency_s":0.496924}2016gitlab # [ 252.000434] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.360Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"82d4c8407bbf40ea1cac2496","created_at":"2026-09-18T22:14:03.848Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:03.849Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-82d4c8407bbf40ea1cac2496: done: 0.013543 sec","job_status":"done","queue_duration_s":0.496911,"scheduling_latency_s":0.496924,"redis_calls":7,"redis_duration_s":0.001051,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013543,"completed_at":"2026-09-18T22:14:04.360Z","load_balancing_strategy":"primary","db_duration_s":0.001874,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017gitlab # [ 252.064160] gitaly[895]: time="2026-09-18T22:14:04.598Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AX6KH53966J0S0AFCHC6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:34.496" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:04.597" grpc.time_ms=0.658 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2018gitlab # [ 252.064834] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.361Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"25ea2a32a0ad46e01c54b717","created_at":"2026-09-18T22:14:04.004Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.006Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-25ea2a32a0ad46e01c54b717: start","job_status":"start","queue_duration_s":0.354804,"scheduling_latency_s":0.354817}2019gitlab # [ 252.065431] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.604Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"25ea2a32a0ad46e01c54b717","created_at":"2026-09-18T22:14:04.004Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.006Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2539,"job_status":"rescheduled","message":"PipelineProcessWorker JID-25ea2a32a0ad46e01c54b717: rescheduled"}2020gitlab # [ 252.110781] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.609Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"25ea2a32a0ad46e01c54b717","created_at":"2026-09-18T22:14:04.004Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-a23631caf8e04fc8","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.006Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-25ea2a32a0ad46e01c54b717: done: 0.248586 sec","job_status":"done","queue_duration_s":0.354804,"scheduling_latency_s":0.354817,"gitaly_calls":3,"gitaly_duration_s":0.134744,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008769,"redis_read_bytes":146,"redis_write_bytes":18015,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000323,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000855,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.001977,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12314,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.003153,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00016,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00061,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001368,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084442,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.225501845,"duration_s":0.248586,"completed_at":"2026-09-18T22:14:04.609Z","load_balancing_strategy":"primary","db_duration_s":0.025966,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2021gitlab # [ 252.152235] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.611Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"50be9446ad141141de1cfe0b","created_at":"2026-09-18T22:14:04.396Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.397Z","job_size_bytes":23,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-50be9446ad141141de1cfe0b: start","job_status":"start","queue_duration_s":0.213532,"scheduling_latency_s":0.213549}2022gitlab # [ 252.152952] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.641Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"50be9446ad141141de1cfe0b","created_at":"2026-09-18T22:14:04.396Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.397Z","job_size_bytes":23,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-50be9446ad141141de1cfe0b: done: 0.030514 sec","job_status":"done","queue_duration_s":0.213532,"scheduling_latency_s":0.213549,"redis_calls":7,"redis_duration_s":0.001534,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000381,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030514,"completed_at":"2026-09-18T22:14:04.641Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008197,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2023gitlab # [ 252.201744] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.643Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e23d0e051a5c19ec045ca1cc","created_at":"2026-09-18T22:14:04.470Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.472Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e23d0e051a5c19ec045ca1cc: start","job_status":"start","queue_duration_s":0.170217,"scheduling_latency_s":0.170231}2024gitlab # [ 252.202689] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.662Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e23d0e051a5c19ec045ca1cc","created_at":"2026-09-18T22:14:04.470Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.472Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e23d0e051a5c19ec045ca1cc: done: 0.019109 sec","job_status":"done","queue_duration_s":0.170217,"scheduling_latency_s":0.170231,"redis_calls":7,"redis_duration_s":0.001468,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019109,"completed_at":"2026-09-18T22:14:04.662Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004701,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2025gitlab # [ 252.247998] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.663Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"89b42713e5c9c2f7554c9e4c","created_at":"2026-09-18T22:14:04.477Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.479Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-89b42713e5c9c2f7554c9e4c: start","job_status":"start","queue_duration_s":0.18375,"scheduling_latency_s":0.183764}2026gitlab # [ 252.248756] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.683Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"89b42713e5c9c2f7554c9e4c","created_at":"2026-09-18T22:14:04.477Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.479Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-89b42713e5c9c2f7554c9e4c: done: 0.019845 sec","job_status":"done","queue_duration_s":0.18375,"scheduling_latency_s":0.183764,"redis_calls":7,"redis_duration_s":0.001461,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019845,"completed_at":"2026-09-18T22:14:04.683Z","load_balancing_strategy":"primary","db_duration_s":0.004969,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2027gitlab # [ 252.277142] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.684Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8c6098ece4e7f9714b5e3176","created_at":"2026-09-18T22:14:04.485Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.486Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8c6098ece4e7f9714b5e3176: start","job_status":"start","queue_duration_s":0.197373,"scheduling_latency_s":0.197388}2028gitlab # [ 252.291597] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.706Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8c6098ece4e7f9714b5e3176","created_at":"2026-09-18T22:14:04.485Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.486Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8c6098ece4e7f9714b5e3176: done: 0.022043 sec","job_status":"done","queue_duration_s":0.197373,"scheduling_latency_s":0.197388,"redis_calls":10,"redis_duration_s":0.002053,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.0006,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022043,"completed_at":"2026-09-18T22:14:04.706Z","load_balancing_strategy":"primary","db_duration_s":0.005172,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2029gitlab # [ 252.354750] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.707Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"594c8d79c4b1fbec8b49d280","created_at":"2026-09-18T22:14:04.487Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.488Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-594c8d79c4b1fbec8b49d280: start","job_status":"start","queue_duration_s":0.219308,"scheduling_latency_s":0.219323}2030gitlab # [ 252.355614] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.726Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"594c8d79c4b1fbec8b49d280","created_at":"2026-09-18T22:14:04.487Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.488Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-594c8d79c4b1fbec8b49d280: done: 0.018529 sec","job_status":"done","queue_duration_s":0.219308,"scheduling_latency_s":0.219323,"redis_calls":7,"redis_duration_s":0.00179,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000414,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018529,"completed_at":"2026-09-18T22:14:04.726Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004268,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2031gitlab # [ 252.385636] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.727Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"03eb3af6e96ed67ec9d55377","created_at":"2026-09-18T22:14:04.488Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.489Z","job_size_bytes":19,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-03eb3af6e96ed67ec9d55377: start","job_status":"start","queue_duration_s":0.237285,"scheduling_latency_s":0.237298}2032gitlab # [ 252.433120] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.745Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"03eb3af6e96ed67ec9d55377","created_at":"2026-09-18T22:14:04.488Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.489Z","job_size_bytes":19,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-03eb3af6e96ed67ec9d55377: done: 0.017894 sec","job_status":"done","queue_duration_s":0.237285,"scheduling_latency_s":0.237298,"redis_calls":8,"redis_duration_s":0.0017009999999999998,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001295,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017894,"completed_at":"2026-09-18T22:14:04.744Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2033gitlab # [ 252.438413] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.746Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"30d0136f8939f215229b3d12","created_at":"2026-09-18T22:14:04.490Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.491Z","job_size_bytes":66,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-30d0136f8939f215229b3d12: start","job_status":"start","queue_duration_s":0.254654,"scheduling_latency_s":0.254667}2034gitlab # [ 252.454110] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.761Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"30d0136f8939f215229b3d12","created_at":"2026-09-18T22:14:04.490Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.491Z","job_size_bytes":66,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-30d0136f8939f215229b3d12: done: 0.015215 sec","job_status":"done","queue_duration_s":0.254654,"scheduling_latency_s":0.254667,"redis_calls":7,"redis_duration_s":0.001444,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015215,"completed_at":"2026-09-18T22:14:04.761Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002773,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2035gitlab # [ 252.511165] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.762Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b060691a93158f0ff1e849e0","created_at":"2026-09-18T22:14:04.491Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.492Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b060691a93158f0ff1e849e0: start","job_status":"start","queue_duration_s":0.269724,"scheduling_latency_s":0.269738}2036gitlab # [ 252.511857] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.784Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b060691a93158f0ff1e849e0","created_at":"2026-09-18T22:14:04.491Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.492Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b060691a93158f0ff1e849e0: done: 0.021902 sec","job_status":"done","queue_duration_s":0.269724,"scheduling_latency_s":0.269738,"redis_calls":7,"redis_duration_s":0.001542,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021902,"completed_at":"2026-09-18T22:14:04.784Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2037gitlab # [ 252.546495] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.785Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"b0acbbd2b293784ce310340f","created_at":"2026-09-18T22:14:04.493Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.494Z","job_size_bytes":6,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-b0acbbd2b293784ce310340f: start","job_status":"start","queue_duration_s":0.291125,"scheduling_latency_s":0.291139}2038gitlab # [ 252.560678] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.795Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"b0acbbd2b293784ce310340f","created_at":"2026-09-18T22:14:04.493Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.494Z","job_size_bytes":6,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-b0acbbd2b293784ce310340f: done: 0.00938 sec","job_status":"done","queue_duration_s":0.291125,"scheduling_latency_s":0.291139,"redis_calls":7,"redis_duration_s":0.0013650000000000001,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000381,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.00938,"completed_at":"2026-09-18T22:14:04.795Z","load_balancing_strategy":"primary","db_duration_s":0.000443,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2039gitlab # [ 252.603167] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.796Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f47da002a493b88c19eaa45f","created_at":"2026-09-18T22:14:04.494Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.495Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f47da002a493b88c19eaa45f: start","job_status":"start","queue_duration_s":0.300148,"scheduling_latency_s":0.300162}2040gitlab # [ 252.649545] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.812Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f47da002a493b88c19eaa45f","created_at":"2026-09-18T22:14:04.494Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.495Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f47da002a493b88c19eaa45f: done: 0.015965 sec","job_status":"done","queue_duration_s":0.300148,"scheduling_latency_s":0.300162,"redis_calls":7,"redis_duration_s":0.0014860000000000001,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000376,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015965,"completed_at":"2026-09-18T22:14:04.812Z","load_balancing_strategy":"primary","db_duration_s":0.003255,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2041gitlab # [ 252.650749] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.813Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"53deb5d6d7c8be184d5106cb","created_at":"2026-09-18T22:14:04.605Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.606Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-53deb5d6d7c8be184d5106cb: start","job_status":"start","queue_duration_s":0.206755,"scheduling_latency_s":0.206768}2042gitlab # [ 252.663689] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:04.830Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"53deb5d6d7c8be184d5106cb","created_at":"2026-09-18T22:14:04.605Z","trace_propagation_headers":{"sentry-trace":"b195d1736bec4e6babd89b8a7432cc5b-3aa7da422a534053","baggage":"sentry-trace_id=b195d1736bec4e6babd89b8a7432cc5b,sentry-release=Unknown"},"correlation_id":"01M2V9AX6KH53966J0S0AFCHC6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:04.606Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-53deb5d6d7c8be184d5106cb: done: 0.017433 sec","job_status":"done","queue_duration_s":0.206755,"scheduling_latency_s":0.206768,"redis_calls":8,"redis_duration_s":0.001682,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000375,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017433,"completed_at":"2026-09-18T22:14:04.830Z","load_balancing_strategy":"primary","db_duration_s":0.003723,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2043gitlab # [ 252.743389] nixbot[2660]: {"ts": "2026-09-18T22:14:05.480Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6057835189e4eae24eaf6d1c6c6f0a70eb7d655b \"HTTP/1.1 201 Created\""}2044gitlab # [ 252.747579] nixbot[2660]: INFO: ::ffff:127.0.0.1:42310 - "GET /api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 200 OK2045gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131', in 1.56 seconds)2046gitlab # [ 252.918542] gitaly[895]: time="2026-09-18T22:14:05.656Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:35.654" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:05.654" grpc.time_ms=1.022 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2047gitlab # [ 252.942745] gitaly[895]: time="2026-09-18T22:14:05.680Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-18T22:14:35.673" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:05.674" grpc.time_ms=6.675 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2048gitlab # [ 252.958829] gitaly[895]: time="2026-09-18T22:14:05.696Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:15.019" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:05.689" grpc.time_ms=7.578 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2049gitlab # [ 252.987309] gitaly[895]: time="2026-09-18T22:14:05.725Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-18T22:14:15.054" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:14:05.725" grpc.time_ms=0.237 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2050gitlab # [ 253.009234] gitaly[895]: time="2026-09-18T22:14:05.747Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-18T22:14:15.068" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:05.739" grpc.time_ms=7.688 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2051gitlab # [ 253.057898] gitaly[895]: time="2026-09-18T22:14:05.796Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:05.795" grpc.time_ms=0.317 pid=895 span.kind=server system=grpc2052gitlab # [ 253.094826] gitaly[895]: time="2026-09-18T22:14:05.832Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:05.832" grpc.time_ms=0.333 pid=895 span.kind=server system=grpc2053gitlab # [ 253.109261] gitaly[895]: time="2026-09-18T22:14:05.847Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:14:15.089" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:14:05.760" operations="[repack_refs]" pid=895 ref=refs/keep-around/789e09c2c59589392a122d7de0d22c17cad48131 remote_ip="2001:db8:1::1" revision=789e09c2c59589392a122d7de0d22c17cad48131 span.kind=server system=grpc username=root2054gitlab # [ 253.116671] gitaly[895]: time="2026-09-18T22:14:05.854Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN diskcache=128e8809-f8a2-42f2-80cb-db52411ebc23 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:14:15.089" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:14:05.760" pid=895 ref=refs/keep-around/789e09c2c59589392a122d7de0d22c17cad48131 remote_ip="2001:db8:1::1" revision=789e09c2c59589392a122d7de0d22c17cad48131 span.kind=server system=grpc username=root2055gitlab # [ 253.117626] gitaly[895]: time="2026-09-18T22:14:05.855Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=90 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5547 command.oublock=8 command.real_time_ms=85 command.spawn_token_fork_ms=0 command.system_time_ms=53 command.user_time_ms=37 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:14:15.089" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:14:05.760" grpc.time_ms=94.706 pid=895 ref=refs/keep-around/789e09c2c59589392a122d7de0d22c17cad48131 remote_ip="2001:db8:1::1" revision=789e09c2c59589392a122d7de0d22c17cad48131 span.kind=server system=grpc username=root2056gitlab # [ 253.121258] gitaly[895]: time="2026-09-18T22:14:05.859Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:35.858" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:05.858" grpc.time_ms=0.518 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2057gitlab # [ 253.129397] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:05.866Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e597860b75ad4128af6b143c","created_at":"2026-09-18T22:14:05.861Z","trace_propagation_headers":{"sentry-trace":"09831935b37146b99461ba23e9fa72b7-b62c552544af4d55","baggage":"sentry-trace_id=09831935b37146b99461ba23e9fa72b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AZG4AX8NDXVD15T9P7EN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:05.864Z","job_size_bytes":65,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e597860b75ad4128af6b143c: start","job_status":"start","queue_duration_s":0.001628,"scheduling_latency_s":0.001689}2058gitlab # [ 253.164910] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:05.902Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"e597860b75ad4128af6b143c","created_at":"2026-09-18T22:14:05.861Z","trace_propagation_headers":{"sentry-trace":"09831935b37146b99461ba23e9fa72b7-b62c552544af4d55","baggage":"sentry-trace_id=09831935b37146b99461ba23e9fa72b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AZG4AX8NDXVD15T9P7EN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:05.864Z","job_size_bytes":65,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-e597860b75ad4128af6b143c: done: 0.036467 sec","job_status":"done","queue_duration_s":0.001628,"scheduling_latency_s":0.001689,"redis_calls":7,"redis_duration_s":0.001337,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000384,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036467,"completed_at":"2026-09-18T22:14:05.902Z","load_balancing_strategy":"primary","db_duration_s":0.005837,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2059gitlab # [ 253.169678] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:05.907Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"dbef7953f26121a92d5e439f","created_at":"2026-09-18T22:14:05.866Z","trace_propagation_headers":{"sentry-trace":"09831935b37146b99461ba23e9fa72b7-b62c552544af4d55","baggage":"sentry-trace_id=09831935b37146b99461ba23e9fa72b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AZG4AX8NDXVD15T9P7EN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:05.870Z","job_size_bytes":65,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-dbef7953f26121a92d5e439f: start","job_status":"start","queue_duration_s":0.036916,"scheduling_latency_s":0.036933}2060gitlab # [ 253.207197] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 3702061gitlab # [ 253.207933] nixbot[2660]: {"ts": "2026-09-18T22:14:05.943Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/789e09c2c59589392a122d7de0d22c17cad48131 \"HTTP/1.1 201 Created\""}2062gitlab # [ 253.213905] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:05.951Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"dbef7953f26121a92d5e439f","created_at":"2026-09-18T22:14:05.866Z","trace_propagation_headers":{"sentry-trace":"09831935b37146b99461ba23e9fa72b7-b62c552544af4d55","baggage":"sentry-trace_id=09831935b37146b99461ba23e9fa72b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AZG4AX8NDXVD15T9P7EN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:05.870Z","job_size_bytes":65,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-dbef7953f26121a92d5e439f: done: 0.044134 sec","job_status":"done","queue_duration_s":0.036916,"scheduling_latency_s":0.036933,"redis_calls":10,"redis_duration_s":0.00154,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000707,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044134,"completed_at":"2026-09-18T22:14:05.951Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013976,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063gitlab # [ 253.254797] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:05.953Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8b3b43d7f67f9236decc6015","created_at":"2026-09-18T22:14:05.928Z","trace_propagation_headers":{"sentry-trace":"09831935b37146b99461ba23e9fa72b7-b62c552544af4d55","baggage":"sentry-trace_id=09831935b37146b99461ba23e9fa72b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AZG4AX8NDXVD15T9P7EN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:05.930Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8b3b43d7f67f9236decc6015: start","job_status":"start","queue_duration_s":0.022813,"scheduling_latency_s":0.022872}2064gitlab # [ 253.255578] nixbot[2660]: {"ts": "2026-09-18T22:14:05.952Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-789e09c2c595-5"}2065gitlab # [ 253.266120] gitaly[895]: time="2026-09-18T22:14:06.004Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-18T22:14:36.002" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:06.003" grpc.time_ms=0.743 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2066gitlab # [ 253.289780] gitaly[895]: time="2026-09-18T22:14:06.027Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:36.026" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:06.027" grpc.time_ms=0.562 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 253.299151] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:06.035Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8b3b43d7f67f9236decc6015","created_at":"2026-09-18T22:14:05.928Z","trace_propagation_headers":{"sentry-trace":"09831935b37146b99461ba23e9fa72b7-b62c552544af4d55","baggage":"sentry-trace_id=09831935b37146b99461ba23e9fa72b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2V9AZG4AX8NDXVD15T9P7EN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:05.930Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8b3b43d7f67f9236decc6015: done: 0.082844 sec","job_status":"done","queue_duration_s":0.022813,"scheduling_latency_s":0.022872,"gitaly_calls":2,"gitaly_duration_s":0.007422,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004035,"redis_read_bytes":79,"redis_write_bytes":5120,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00059,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000615,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000189,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1131,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00017,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000284,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000643,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056284,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.060605468,"duration_s":0.082844,"completed_at":"2026-09-18T22:14:06.035Z","load_balancing_strategy":"primary","db_duration_s":0.016507,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2068gitlab # [ 253.347742] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:06.037Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"eae76c85cdde3506ee120cef","created_at":"2026-09-18T22:14:05.992Z","trace_propagation_headers":{"sentry-trace":"09831935b37146b99461ba23e9fa72b7-a640ec580df74da3","baggage":"sentry-trace_id=09831935b37146b99461ba23e9fa72b7,sentry-release=Unknown"},"correlation_id":"01M2V9AZG4AX8NDXVD15T9P7EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:05.993Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-eae76c85cdde3506ee120cef: start","job_status":"start","queue_duration_s":0.043571,"scheduling_latency_s":0.043588}2069gitlab # [ 253.348510] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:06.058Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"eae76c85cdde3506ee120cef","created_at":"2026-09-18T22:14:05.992Z","trace_propagation_headers":{"sentry-trace":"09831935b37146b99461ba23e9fa72b7-a640ec580df74da3","baggage":"sentry-trace_id=09831935b37146b99461ba23e9fa72b7,sentry-release=Unknown"},"correlation_id":"01M2V9AZG4AX8NDXVD15T9P7EN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:05.993Z","job_size_bytes":3,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-eae76c85cdde3506ee120cef: done: 0.021641 sec","job_status":"done","queue_duration_s":0.043571,"scheduling_latency_s":0.043588,"redis_calls":7,"redis_duration_s":0.0010270000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021641,"completed_at":"2026-09-18T22:14:06.058Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003357,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070gitlab # [ 253.391695] nixbot[2660]: {"ts": "2026-09-18T22:14:06.047Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-789e09c2c595-5", "argv0": "bwrap"}2071gitlab # [ 253.406420] gitaly[895]: time="2026-09-18T22:14:06.144Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:14:15.089" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:14:05.760" pid=895 ref=refs/keep-around/789e09c2c59589392a122d7de0d22c17cad48131 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-18 22:13:05.27701242 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" revision=789e09c2c59589392a122d7de0d22c17cad48131 span.kind=server system=grpc username=root2072gitlab # [ 253.430179] gitaly[895]: time="2026-09-18T22:14:06.168Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:14:15.089" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:14:05.760" pid=895 ref=refs/keep-around/789e09c2c59589392a122d7de0d22c17cad48131 remote_ip="2001:db8:1::1" revision=789e09c2c59589392a122d7de0d22c17cad48131 span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2073gitlab # [ 253.430419] gitaly[895]: time="2026-09-18T22:14:06.168Z" level=info msg="optimized repository" clean_stale_data_time=23.447114ms clean_work_trees_time="69.282µs" commit_graph_time="3.352µs" correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.911µs" pid=895 prune_time="3.352µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.75µs" storage_name=default system=housekeeping total_time=23.641273ms2074gitlab # [ 253.430882] gitaly[895]: time="2026-09-18T22:14:06.168Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9AZG4AX8NDXVD15T9P7EN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-18T22:14:15.089" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-18T22:14:05.760" pid=895 ref=refs/keep-around/789e09c2c59589392a122d7de0d22c17cad48131 remote_ip="2001:db8:1::1" revision=789e09c2c59589392a122d7de0d22c17cad48131 span.kind=server system=grpc username=root2075gitlab # [ 253.461823] nix-daemon[2820]: accepted connection from pid 3673, user nixbot2076gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131'2077gitlab # [ 253.806896] nixbot[2660]: INFO: ::ffff:127.0.0.1:59014 - "GET /api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 200 OK2078gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=789e09c2c59589392a122d7de0d22c17cad48131', in 0.02 seconds)2079(finished: subtest: push delivers the webhook, in 50.97 seconds)2080subtest: eval and build post commit statuses2081gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses2082gitlab # [ 253.858329] gitaly[895]: time="2026-09-18T22:14:06.596Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B0F37EWRY2Y7H1JZ5663 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:36.595" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:06.595" grpc.time_ms=0.545 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2083gitlab # [ 253.887361] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 622084gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses, in 0.08 seconds)2085gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses2086gitlab # [ 254.982908] gitaly[895]: time="2026-09-18T22:14:07.720Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B1HF9PFC8FH16KJ3GNV2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:37.718" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:07.718" grpc.time_ms=0.998 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2087gitlab # [ 255.009237] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 822088gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses, in 0.13 seconds)2089gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses2090gitlab # [ 256.100745] gitaly[895]: time="2026-09-18T22:14:08.838Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B2MKH3YE7EQT8T0XDHGT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:38.837" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:08.837" grpc.time_ms=0.655 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2091gitlab # [ 256.160605] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1102092gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses, in 0.15 seconds)2093gitlab # [ 257.128900] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:09.866Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"11dbb09badeb9756194e883a","created_at":"2026-09-18T22:14:09.856Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"705d4068b57f135cff3c74043d48a784","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.864Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-11dbb09badeb9756194e883a: start","job_status":"start","queue_duration_s":0.001677,"scheduling_latency_s":0.001725}2094gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses2095gitlab # [ 257.231871] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:09.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"11dbb09badeb9756194e883a","created_at":"2026-09-18T22:14:09.856Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"705d4068b57f135cff3c74043d48a784","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.864Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-11dbb09badeb9756194e883a: done: 0.103341 sec","job_status":"done","queue_duration_s":0.001677,"scheduling_latency_s":0.001725,"redis_calls":11,"redis_duration_s":0.009638,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003808,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004544,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000406755,"exclusive_lock_hold_duration_s":0.041720945,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.103341,"completed_at":"2026-09-18T22:14:09.969Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018264,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2096gitlab # [ 257.278456] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:09.973Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c35c46aafd17f9f7583c64cc","created_at":"2026-09-18T22:14:09.870Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a70d3d4b8c0c096af3b8e61fc6418f75","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.878Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c35c46aafd17f9f7583c64cc: start","job_status":"start","queue_duration_s":0.095432,"scheduling_latency_s":0.095461}2097gitlab # [ 257.278916] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.008Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c35c46aafd17f9f7583c64cc","created_at":"2026-09-18T22:14:09.870Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a70d3d4b8c0c096af3b8e61fc6418f75","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.878Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c35c46aafd17f9f7583c64cc: done: 0.034943 sec","job_status":"done","queue_duration_s":0.095432,"scheduling_latency_s":0.095461,"redis_calls":11,"redis_duration_s":0.003288,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000806,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001314,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001168,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000409828,"exclusive_lock_hold_duration_s":0.004623772,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.034943,"completed_at":"2026-09-18T22:14:10.008Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004634,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2098gitlab # [ 257.320056] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.009Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f869983cc61325f0d0ad0d11","created_at":"2026-09-18T22:14:09.883Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4024472a6abd966152c2f175bd7b451","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.911Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f869983cc61325f0d0ad0d11: start","job_status":"start","queue_duration_s":0.098738,"scheduling_latency_s":0.098752}2099gitlab # [ 257.357922] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.023Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2100gitlab # [ 257.358720] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.030Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f869983cc61325f0d0ad0d11","created_at":"2026-09-18T22:14:09.883Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4024472a6abd966152c2f175bd7b451","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.911Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f869983cc61325f0d0ad0d11: done: 0.021024 sec","job_status":"done","queue_duration_s":0.098738,"scheduling_latency_s":0.098752,"redis_calls":9,"redis_duration_s":0.003491,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000515,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002205,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000771,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021024,"completed_at":"2026-09-18T22:14:10.030Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000503,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2101gitlab # [ 257.397232] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"886e7c736c3845f2b028209f","created_at":"2026-09-18T22:14:09.932Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57300f32bd39d50fedc438089d7efbef","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.942Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-886e7c736c3845f2b028209f: start","job_status":"start","queue_duration_s":0.090869,"scheduling_latency_s":0.090886}2102gitlab # [ 257.397953] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"886e7c736c3845f2b028209f","created_at":"2026-09-18T22:14:09.932Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57300f32bd39d50fedc438089d7efbef","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.942Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-886e7c736c3845f2b028209f: done: 0.082639 sec","job_status":"done","queue_duration_s":0.090869,"scheduling_latency_s":0.090886,"redis_calls":9,"redis_duration_s":0.049061,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000983,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.047233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082639,"completed_at":"2026-09-18T22:14:10.115Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004365,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2103gitlab # [ 257.412173] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d56c5ba16b726e1efe4603b6","created_at":"2026-09-18T22:14:09.964Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a53eeef22f8f63c1c7b02bf08a012d3a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.972Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d56c5ba16b726e1efe4603b6: start","job_status":"start","queue_duration_s":0.1465,"scheduling_latency_s":0.146532}2104gitlab # [ 257.424817] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.147Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2105gitlab # [ 257.427589] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.163Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d56c5ba16b726e1efe4603b6","created_at":"2026-09-18T22:14:09.964Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a53eeef22f8f63c1c7b02bf08a012d3a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.972Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d56c5ba16b726e1efe4603b6: done: 0.044314 sec","job_status":"done","queue_duration_s":0.1465,"scheduling_latency_s":0.146532,"redis_calls":9,"redis_duration_s":0.00872,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001831,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005373,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001516,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044314,"completed_at":"2026-09-18T22:14:10.163Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106gitlab # [ 257.472964] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"32d299aedea5aba5e0f5aba3","created_at":"2026-09-18T22:14:09.982Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22f7e1c95f48676d2013ebdeb3e55436","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.991Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-32d299aedea5aba5e0f5aba3: start","job_status":"start","queue_duration_s":0.174489,"scheduling_latency_s":0.174512}2107gitlab # [ 257.517188] gitaly[895]: time="2026-09-18T22:14:10.254Z" level=info msg="finished unary call with code OK" catfile.duration_ms=260 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=260 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B3R7FE11RWV8BVQA91MP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:39.093" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:09.993" grpc.time_ms=261.318 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2108gitlab # [ 257.561095] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 3702109gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses, in 0.40 seconds)2110gitlab # [ 257.933076] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.670Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"32d299aedea5aba5e0f5aba3","created_at":"2026-09-18T22:14:09.982Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22f7e1c95f48676d2013ebdeb3e55436","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:09.991Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-32d299aedea5aba5e0f5aba3: done: 0.505161 sec","job_status":"done","queue_duration_s":0.174489,"scheduling_latency_s":0.174512,"redis_calls":564,"redis_duration_s":0.271895,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001171,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.016887,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.253837,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.264589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.505161,"completed_at":"2026-09-18T22:14:10.670Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2111gitlab # [ 257.970951] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.708Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c8ace3a90a7131acb2df87f2","created_at":"2026-09-18T22:14:09.998Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f1dbb1bf2da016ea33529d988ecdda1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.007Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c8ace3a90a7131acb2df87f2: start","job_status":"start","queue_duration_s":0.700865,"scheduling_latency_s":0.700893}2112gitlab # [ 257.993474] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.731Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2113gitlab # [ 258.007839] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.744Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c8ace3a90a7131acb2df87f2","created_at":"2026-09-18T22:14:09.998Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f1dbb1bf2da016ea33529d988ecdda1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.007Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c8ace3a90a7131acb2df87f2: done: 0.035933 sec","job_status":"done","queue_duration_s":0.700865,"scheduling_latency_s":0.700893,"redis_calls":9,"redis_duration_s":0.003479,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002839,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000398,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035933,"completed_at":"2026-09-18T22:14:10.744Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000602,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2114gitlab # [ 258.017139] nix-daemon[2820]: accepted connection from pid 3717, user nixbot2115gitlab # [ 258.047837] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.785Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"120ba3753c029bebbdd05135","created_at":"2026-09-18T22:14:10.012Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"276927e53639ae0f536526f3c84e8f05","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.019Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-120ba3753c029bebbdd05135: start","job_status":"start","queue_duration_s":0.766043,"scheduling_latency_s":0.766082}2116gitlab # [ 258.059440] nix-daemon[2820]: accepted connection from pid 3673, user nixbot2117gitlab # [ 258.071569] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.809Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"120ba3753c029bebbdd05135","created_at":"2026-09-18T22:14:10.012Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"276927e53639ae0f536526f3c84e8f05","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.019Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-120ba3753c029bebbdd05135: done: 0.023854 sec","job_status":"done","queue_duration_s":0.766043,"scheduling_latency_s":0.766082,"redis_calls":11,"redis_duration_s":0.002027,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000318,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000387,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000377,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023854,"completed_at":"2026-09-18T22:14:10.809Z","load_balancing_strategy":"primary","db_duration_s":0.001509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118gitlab # [ 258.093559] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.810Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"cde8b62f141940d68a19adcc","created_at":"2026-09-18T22:14:10.024Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"094844f05af22b78b2cefb21d8182fb7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.031Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-cde8b62f141940d68a19adcc: start","job_status":"start","queue_duration_s":0.778824,"scheduling_latency_s":0.778847}2119gitlab # [ 258.106425] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.844Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"cde8b62f141940d68a19adcc","created_at":"2026-09-18T22:14:10.024Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"094844f05af22b78b2cefb21d8182fb7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.031Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-cde8b62f141940d68a19adcc: done: 0.033615 sec","job_status":"done","queue_duration_s":0.778824,"scheduling_latency_s":0.778847,"redis_calls":13,"redis_duration_s":0.001909,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000537,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000621,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000421562,"exclusive_lock_hold_duration_s":0.015512865,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.033615,"completed_at":"2026-09-18T22:14:10.844Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007744,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2120gitlab # [ 258.153995] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.845Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"2ead3c8443a9c708db0c0334","created_at":"2026-09-18T22:14:10.037Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"beadb60bd67571f623078142be4c0776","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.041Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-2ead3c8443a9c708db0c0334: start","job_status":"start","queue_duration_s":0.803995,"scheduling_latency_s":0.804012}2121gitlab # [ 258.154731] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"2ead3c8443a9c708db0c0334","created_at":"2026-09-18T22:14:10.037Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"beadb60bd67571f623078142be4c0776","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.041Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-2ead3c8443a9c708db0c0334: done: 0.017856 sec","job_status":"done","queue_duration_s":0.803995,"scheduling_latency_s":0.804012,"redis_calls":11,"redis_duration_s":0.001386,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000628,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000492,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000322108,"exclusive_lock_hold_duration_s":0.003593753,"duration_s":0.017856,"completed_at":"2026-09-18T22:14:10.863Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122gitlab # [ 258.202312] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.864Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"0e80e1ef0dc04344d9b4b0df","created_at":"2026-09-18T22:14:10.049Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a972b63b52349c44bd231dcfe95f0e3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.056Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-0e80e1ef0dc04344d9b4b0df: start","job_status":"start","queue_duration_s":0.807969,"scheduling_latency_s":0.807984}2123gitlab # [ 258.214478] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.952Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"0e80e1ef0dc04344d9b4b0df","created_at":"2026-09-18T22:14:10.049Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a972b63b52349c44bd231dcfe95f0e3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.056Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-0e80e1ef0dc04344d9b4b0df: done: 0.088125 sec","job_status":"done","queue_duration_s":0.807969,"scheduling_latency_s":0.807984,"redis_calls":12,"redis_duration_s":0.003686,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000224,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002507,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00029473,"exclusive_lock_hold_duration_s":0.072474574,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.088125,"completed_at":"2026-09-18T22:14:10.952Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004406,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}2124gitlab # [ 258.273915] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.953Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fa4c7cff70d7a94cdb962759","created_at":"2026-09-18T22:14:10.111Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e21cbac96178d35e3aa4c597fe004b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-18T22:14:10.117Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fa4c7cff70d7a94cdb962759: start","job_status":"start","queue_duration_s":0.835427,"scheduling_latency_s":0.835442}2125gitlab # [ 258.274625] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.961Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2126gitlab # [ 258.275243] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.966Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fa4c7cff70d7a94cdb962759","created_at":"2026-09-18T22:14:10.111Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e21cbac96178d35e3aa4c597fe004b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-18T22:14:10.117Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fa4c7cff70d7a94cdb962759: done: 0.013126 sec","job_status":"done","queue_duration_s":0.835427,"scheduling_latency_s":0.835442,"redis_calls":9,"redis_duration_s":0.001264,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013126,"completed_at":"2026-09-18T22:14:10.966Z","load_balancing_strategy":"primary","db_duration_s":0.000543,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2127gitlab # [ 258.333693] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:10.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"05f9da72af5190ee7d8e439b","created_at":"2026-09-18T22:14:10.122Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c2fed735b64c9729a8bb443fbfb13d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.126Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-05f9da72af5190ee7d8e439b: start","job_status":"start","queue_duration_s":0.841292,"scheduling_latency_s":0.841307}2128gitlab # [ 258.355377] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.093Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"05f9da72af5190ee7d8e439b","created_at":"2026-09-18T22:14:10.122Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c2fed735b64c9729a8bb443fbfb13d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.126Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-05f9da72af5190ee7d8e439b: done: 0.125723 sec","job_status":"done","queue_duration_s":0.841292,"scheduling_latency_s":0.841307,"redis_calls":13,"redis_duration_s":0.0017929999999999999,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000486,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000638,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000306743,"exclusive_lock_hold_duration_s":0.109780991,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.125723,"completed_at":"2026-09-18T22:14:11.093Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2129gitlab # [ 258.356636] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.094Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1daf99c962322f6ecbd4c86c","created_at":"2026-09-18T22:14:10.133Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95ea9800f85b69bb2f888fee6bb8dd69","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.148Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1daf99c962322f6ecbd4c86c: start","job_status":"start","queue_duration_s":0.9458,"scheduling_latency_s":0.945817}2130gitlab # [ 258.397683] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.116Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1daf99c962322f6ecbd4c86c","created_at":"2026-09-18T22:14:10.133Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95ea9800f85b69bb2f888fee6bb8dd69","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.148Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1daf99c962322f6ecbd4c86c: done: 0.022339 sec","job_status":"done","queue_duration_s":0.9458,"scheduling_latency_s":0.945817,"redis_calls":9,"redis_duration_s":0.001941,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000425,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022339,"completed_at":"2026-09-18T22:14:11.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2131gitlab # [ 258.408967] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"58df08565aa0f44c83dff947","created_at":"2026-09-18T22:14:10.160Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"372c5d0c7310902c3a30619533913d08","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.168Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-58df08565aa0f44c83dff947: start","job_status":"start","queue_duration_s":0.949449,"scheduling_latency_s":0.949463}2132gitlab # [ 258.422731] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.144Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"58df08565aa0f44c83dff947","created_at":"2026-09-18T22:14:10.160Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"372c5d0c7310902c3a30619533913d08","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.168Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-58df08565aa0f44c83dff947: done: 0.026615 sec","job_status":"done","queue_duration_s":0.949449,"scheduling_latency_s":0.949463,"redis_calls":9,"redis_duration_s":0.001873,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000391,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000391,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.026615,"completed_at":"2026-09-18T22:14:11.144Z","load_balancing_strategy":"primary","db_duration_s":0.011087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133gitlab # [ 258.471265] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"59990884b9d2dbd4e6906889","created_at":"2026-09-18T22:14:10.192Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6852520bb716565a0f8618a68ab82fa4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-18T22:14:10.201Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-59990884b9d2dbd4e6906889: start","job_status":"start","queue_duration_s":0.944565,"scheduling_latency_s":0.944582}2134gitlab # [ 258.475765] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.155Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2135gitlab # [ 258.479029] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.159Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"59990884b9d2dbd4e6906889","created_at":"2026-09-18T22:14:10.192Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6852520bb716565a0f8618a68ab82fa4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-18T22:14:10.201Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-59990884b9d2dbd4e6906889: done: 0.013806 sec","job_status":"done","queue_duration_s":0.944565,"scheduling_latency_s":0.944582,"redis_calls":9,"redis_duration_s":0.001799,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00046,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013806,"completed_at":"2026-09-18T22:14:11.159Z","load_balancing_strategy":"primary","db_duration_s":0.000445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136gitlab # [ 258.543255] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.161Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d559874ac8347083e3198aef","created_at":"2026-09-18T22:14:10.209Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.216Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d559874ac8347083e3198aef: start","job_status":"start","queue_duration_s":0.944796,"scheduling_latency_s":0.944812}2137gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses2138gitlab # [ 258.543982] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.190Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d559874ac8347083e3198aef","created_at":"2026-09-18T22:14:10.209Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.216Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d559874ac8347083e3198aef: done: 0.029606 sec","job_status":"done","queue_duration_s":0.944796,"scheduling_latency_s":0.944812,"redis_calls":18,"redis_duration_s":0.004089,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000864,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000281,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002287,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000657,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029606,"completed_at":"2026-09-18T22:14:11.190Z","load_balancing_strategy":"primary","db_duration_s":0.002814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139gitlab # [ 258.566769] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"eb9b81f800fd6a7bde83eaf2","created_at":"2026-09-18T22:14:10.225Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2901ad6b1b7a7530f08d23c33ccd84f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.232Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-eb9b81f800fd6a7bde83eaf2: start","job_status":"start","queue_duration_s":0.959673,"scheduling_latency_s":0.959688}2140gitlab # [ 258.606593] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"eb9b81f800fd6a7bde83eaf2","created_at":"2026-09-18T22:14:10.225Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2901ad6b1b7a7530f08d23c33ccd84f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.232Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-eb9b81f800fd6a7bde83eaf2: done: 0.0191 sec","job_status":"done","queue_duration_s":0.959673,"scheduling_latency_s":0.959688,"redis_calls":9,"redis_duration_s":0.001873,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000391,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0191,"completed_at":"2026-09-18T22:14:11.211Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141gitlab # [ 258.647660] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.212Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b6756fd5ddac1c22fe6c575e","created_at":"2026-09-18T22:14:10.238Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5d3365f4c992a804e63e9ca63bf499a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.244Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b6756fd5ddac1c22fe6c575e: start","job_status":"start","queue_duration_s":0.967312,"scheduling_latency_s":0.967328}2142gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses, in 0.12 seconds)2143gitlab # [ 258.648522] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.232Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b6756fd5ddac1c22fe6c575e","created_at":"2026-09-18T22:14:10.238Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5d3365f4c992a804e63e9ca63bf499a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.244Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b6756fd5ddac1c22fe6c575e: done: 0.020268 sec","job_status":"done","queue_duration_s":0.967312,"scheduling_latency_s":0.967328,"redis_calls":11,"redis_duration_s":0.002311,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001005,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0009,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000390831,"exclusive_lock_hold_duration_s":0.005251505,"duration_s":0.020268,"completed_at":"2026-09-18T22:14:11.232Z","load_balancing_strategy":"primary","db_duration_s":0.003115,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2144gitlab # [ 258.671049] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"85e10c4f15ff079e5d53ea34","created_at":"2026-09-18T22:14:10.336Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41cb3dfc9553db2d78718d3801b4d78f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.347Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-85e10c4f15ff079e5d53ea34: start","job_status":"start","queue_duration_s":0.88647,"scheduling_latency_s":0.886483}2145gitlab # [ 258.680052] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"85e10c4f15ff079e5d53ea34","created_at":"2026-09-18T22:14:10.336Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41cb3dfc9553db2d78718d3801b4d78f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.347Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-85e10c4f15ff079e5d53ea34: done: 0.016048 sec","job_status":"done","queue_duration_s":0.88647,"scheduling_latency_s":0.886483,"redis_calls":11,"redis_duration_s":0.002227,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000809,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016048,"completed_at":"2026-09-18T22:14:11.249Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2146gitlab # [ 258.733891] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.250Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a011b22af9bfdbdce1778893","created_at":"2026-09-18T22:14:10.355Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36744b9d2c87cf5788c39ab94c218062","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.363Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a011b22af9bfdbdce1778893: start","job_status":"start","queue_duration_s":0.887824,"scheduling_latency_s":0.887838}2147gitlab # [ 258.735032] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.258Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2148gitlab # [ 258.739610] gitaly[895]: time="2026-09-18T22:14:11.403Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B554GGQRRA8GYD611EBR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:41.400" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:11.400" grpc.time_ms=3.183 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2149gitlab # [ 258.781726] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 572150gitlab # [ 258.782649] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1272151gitlab # [ 258.783261] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a011b22af9bfdbdce1778893","created_at":"2026-09-18T22:14:10.355Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36744b9d2c87cf5788c39ab94c218062","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.363Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a011b22af9bfdbdce1778893: done: 0.012981 sec","job_status":"done","queue_duration_s":0.887824,"scheduling_latency_s":0.887838,"redis_calls":9,"redis_duration_s":0.001285,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000232,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000305,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012981,"completed_at":"2026-09-18T22:14:11.263Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2152gitlab # [ 258.836583] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.265Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f927e27176697384f785c9ea","created_at":"2026-09-18T22:14:10.379Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.386Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f927e27176697384f785c9ea: start","job_status":"start","queue_duration_s":0.878255,"scheduling_latency_s":0.878268}2153gitlab # [ 258.837243] gitaly[895]: time="2026-09-18T22:14:11.440Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B55J3JGC7NKV20D9KJG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:41.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:11.439" grpc.time_ms=0.527 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2154gitlab # [ 258.838136] nixbot[2660]: {"ts": "2026-09-18T22:14:11.507Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/789e09c2c59589392a122d7de0d22c17cad48131 \"HTTP/1.1 201 Created\""}2155gitlab # [ 258.838611] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f927e27176697384f785c9ea","created_at":"2026-09-18T22:14:10.379Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.386Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f927e27176697384f785c9ea: done: 0.070975 sec","job_status":"done","queue_duration_s":0.878255,"scheduling_latency_s":0.878268,"redis_calls":70,"redis_duration_s":0.011325,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001116,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000489,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.009353,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000367,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.070975,"completed_at":"2026-09-18T22:14:11.335Z","load_balancing_strategy":"primary","db_duration_s":0.001776,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156gitlab # [ 258.886660] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.337Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e9e7406834ecb6cf15a3844a","created_at":"2026-09-18T22:14:10.395Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c89530bb189ad1f867056da58f6151b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.403Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e9e7406834ecb6cf15a3844a: start","job_status":"start","queue_duration_s":0.933695,"scheduling_latency_s":0.933715}2157gitlab # [ 258.887476] gitaly[895]: time="2026-09-18T22:14:11.469Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9B55J3JGC7NKV20D9KJG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:21.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:11.463" grpc.time_ms=5.753 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2158gitlab # [ 258.888676] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 952159gitlab # [ 258.890798] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e9e7406834ecb6cf15a3844a","created_at":"2026-09-18T22:14:10.395Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c89530bb189ad1f867056da58f6151b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.403Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e9e7406834ecb6cf15a3844a: done: 0.012192 sec","job_status":"done","queue_duration_s":0.933695,"scheduling_latency_s":0.933715,"redis_calls":9,"redis_duration_s":0.001175,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000233,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012192,"completed_at":"2026-09-18T22:14:11.349Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000399,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160gitlab # [ 258.959990] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.350Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"12dec1030786ff400ff503e0","created_at":"2026-09-18T22:14:10.409Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34f8a135992b79f9f6048b9e5bf0bf1c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.417Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-12dec1030786ff400ff503e0: start","job_status":"start","queue_duration_s":0.932789,"scheduling_latency_s":0.932803}2161gitlab # [ 258.960711] nixbot[2660]: {"ts": "2026-09-18T22:14:11.603Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/789e09c2c59589392a122d7de0d22c17cad48131 \"HTTP/1.1 201 Created\""}2162gitlab # [ 258.961315] gitaly[895]: time="2026-09-18T22:14:11.547Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B59NZ8D80F8PW016V0KQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:41.546" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:11.546" grpc.time_ms=0.756 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2163gitlab # [ 258.961978] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.369Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"12dec1030786ff400ff503e0","created_at":"2026-09-18T22:14:10.409Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34f8a135992b79f9f6048b9e5bf0bf1c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-18T22:14:10.417Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-12dec1030786ff400ff503e0: done: 0.019359 sec","job_status":"done","queue_duration_s":0.932789,"scheduling_latency_s":0.932803,"redis_calls":12,"redis_duration_s":0.001873,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000768,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005687035,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.019359,"completed_at":"2026-09-18T22:14:11.369Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001744,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2164gitlab # [ 259.003781] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.370Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"88311c68412072c2573d4355","created_at":"2026-09-18T22:14:10.429Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d9c8b2223463d86c8698dd138cdf74d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-18T22:14:10.436Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-88311c68412072c2573d4355: start","job_status":"start","queue_duration_s":0.93413,"scheduling_latency_s":0.934144}2165gitlab # [ 259.007140] gitaly[895]: time="2026-09-18T22:14:11.572Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9B59NZ8D80F8PW016V0KQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:21.237" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:11.567" grpc.time_ms=5.365 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2166gitlab # [ 259.046900] nixbot[2660]: {"ts": "2026-09-18T22:14:11.708Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/789e09c2c59589392a122d7de0d22c17cad48131 \"HTTP/1.1 201 Created\""}2167gitlab # [ 259.047722] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"88311c68412072c2573d4355","created_at":"2026-09-18T22:14:10.429Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-506d9121d9024713","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d9c8b2223463d86c8698dd138cdf74d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-18T22:14:10.436Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-88311c68412072c2573d4355: done: 0.026473 sec","job_status":"done","queue_duration_s":0.93413,"scheduling_latency_s":0.934144,"redis_calls":9,"redis_duration_s":0.003195,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002304,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026473,"completed_at":"2026-09-18T22:14:11.396Z","load_balancing_strategy":"primary","db_duration_s":0.00224,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168gitlab # [ 259.096172] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.399Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"aec2d841d879900283e4d91a","created_at":"2026-09-18T22:14:11.173Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-cd8f6775e1624a36","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.183Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-aec2d841d879900283e4d91a: start","job_status":"start","queue_duration_s":0.215768,"scheduling_latency_s":0.215788}2169gitlab # [ 259.096845] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.410Z","message":"No existing merge request to be cleaned up."}2170gitlab # [ 259.097541] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/789e09c2c59589392a122d7de0d22c17cad48131 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 932171gitlab # [ 259.098065] gitaly[895]: time="2026-09-18T22:14:11.647Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B5CX6QR80CKHJX878BC2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:41.646" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:11.646" grpc.time_ms=0.902 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2172gitlab # [ 259.098637] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.419Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"aec2d841d879900283e4d91a","created_at":"2026-09-18T22:14:11.173Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-cd8f6775e1624a36","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.183Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-aec2d841d879900283e4d91a: done: 0.019841 sec","job_status":"done","queue_duration_s":0.215768,"scheduling_latency_s":0.215788,"redis_calls":12,"redis_duration_s":0.001699,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000851,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019841,"completed_at":"2026-09-18T22:14:11.419Z","load_balancing_strategy":"primary","db_duration_s":0.002066,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2173gitlab # [ 259.147303] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.420Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3071218c7f92d247e3d05914","created_at":"2026-09-18T22:14:11.173Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-cd8f6775e1624a36","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.183Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3071218c7f92d247e3d05914: start","job_status":"start","queue_duration_s":0.236605,"scheduling_latency_s":0.236623}2174gitlab # [ 259.147942] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.428Z","message":"No existing merge request to be cleaned up."}2175gitlab # [ 259.148606] gitaly[895]: time="2026-09-18T22:14:11.674Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9B5CX6QR80CKHJX878BC2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-18T22:14:21.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:11.669" grpc.time_ms=5.338 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2176gitlab # [ 259.149268] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.433Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3071218c7f92d247e3d05914","created_at":"2026-09-18T22:14:11.173Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-cd8f6775e1624a36","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.183Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3071218c7f92d247e3d05914: done: 0.013521 sec","job_status":"done","queue_duration_s":0.236605,"scheduling_latency_s":0.236623,"redis_calls":10,"redis_duration_s":0.001451,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00085,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013521,"completed_at":"2026-09-18T22:14:11.433Z","load_balancing_strategy":"primary","db_duration_s":0.001064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2177gitlab # [ 259.191132] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.434Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"542181c3adef55ecd772a2b0","created_at":"2026-09-18T22:14:11.173Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-cd8f6775e1624a36","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.183Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-542181c3adef55ecd772a2b0: start","job_status":"start","queue_duration_s":0.251125,"scheduling_latency_s":0.251142}2178gitlab # [ 259.191541] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.442Z","message":"No existing merge request to be cleaned up."}2179gitlab # [ 259.192098] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.448Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"542181c3adef55ecd772a2b0","created_at":"2026-09-18T22:14:11.173Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-cd8f6775e1624a36","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.183Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-542181c3adef55ecd772a2b0: done: 0.013603 sec","job_status":"done","queue_duration_s":0.251125,"scheduling_latency_s":0.251142,"redis_calls":10,"redis_duration_s":0.001582,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013603,"completed_at":"2026-09-18T22:14:11.448Z","load_balancing_strategy":"primary","db_duration_s":0.001062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180gitlab # [ 259.234846] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.449Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"504b54f3234b47abe0f2cfd8","created_at":"2026-09-18T22:14:11.173Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-cd8f6775e1624a36","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.183Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-504b54f3234b47abe0f2cfd8: start","job_status":"start","queue_duration_s":0.265866,"scheduling_latency_s":0.265883}2181gitlab # [ 259.237985] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.457Z","message":"No existing merge request to be cleaned up."}2182gitlab # [ 259.243659] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.463Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"504b54f3234b47abe0f2cfd8","created_at":"2026-09-18T22:14:11.173Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-cd8f6775e1624a36","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"959e36b851070a99acb7d726ebea6990","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.183Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-504b54f3234b47abe0f2cfd8: done: 0.014258 sec","job_status":"done","queue_duration_s":0.265866,"scheduling_latency_s":0.265883,"redis_calls":10,"redis_duration_s":0.0014780000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014258,"completed_at":"2026-09-18T22:14:11.463Z","load_balancing_strategy":"primary","db_duration_s":0.001128,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2183gitlab # [ 259.275725] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46bff960338ffd2dbd037bd5","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46bff960338ffd2dbd037bd5: start","job_status":"start","queue_duration_s":0.134989,"scheduling_latency_s":0.135008}2184gitlab # [ 259.286676] gitaly[895]: time="2026-09-18T22:14:11.930Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2V9B5NGK5DMNM7Z6DPTZKKY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:11.925" grpc.time_ms=5.284 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc2185gitlab # [ 259.298624] gitaly[895]: time="2026-09-18T22:14:11.977Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B5PW0CNKBDH8H57MH7S0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:11.969" pid=895 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2186gitlab # [ 259.308099] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:11 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 412187gitlab # [ 259.309974] gitlab-workhorse[828]: time="2026-09-18T22:14:11Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672188gitlab # [ 259.311626] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:11 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 452189gitlab # [ 259.313633] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46bff960338ffd2dbd037bd5","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46bff960338ffd2dbd037bd5: done: 0.015463 sec","job_status":"done","queue_duration_s":0.134989,"scheduling_latency_s":0.135008,"redis_calls":15,"redis_duration_s":0.0020670000000000003,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001301,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015463,"completed_at":"2026-09-18T22:14:11.480Z","load_balancing_strategy":"primary","db_duration_s":0.000996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2190gitlab # [ 259.353539] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f139608814add752f25072c4","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f139608814add752f25072c4: start","job_status":"start","queue_duration_s":0.151363,"scheduling_latency_s":0.151387}2191gitlab # [ 259.397966] gitaly[895]: time="2026-09-18T22:14:11.977Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B5PW0CNKBDH8H57MH7S0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:11.969" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc2192gitlab # [ 259.398675] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/09/18:22:14:12 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 612193gitlab # [ 259.399165] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f139608814add752f25072c4","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f139608814add752f25072c4: done: 0.009918 sec","job_status":"done","queue_duration_s":0.151363,"scheduling_latency_s":0.151387,"redis_calls":13,"redis_duration_s":0.001786,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009918,"completed_at":"2026-09-18T22:14:11.491Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2194gitlab # [ 259.411677] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51b13d43cf2e308431461d3c","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51b13d43cf2e308431461d3c: start","job_status":"start","queue_duration_s":0.16211,"scheduling_latency_s":0.162124}2195gitlab # [ 259.423033] gitaly[895]: time="2026-09-18T22:14:11.977Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B5PW0CNKBDH8H57MH7S0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:11.969" grpc.time_ms=7.511 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc2196gitlab # [ 259.471243] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51b13d43cf2e308431461d3c","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51b13d43cf2e308431461d3c: done: 0.010015 sec","job_status":"done","queue_duration_s":0.16211,"scheduling_latency_s":0.162124,"redis_calls":13,"redis_duration_s":0.001828,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010015,"completed_at":"2026-09-18T22:14:11.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197gitlab # [ 259.472253] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b36c50c2891b887f03de5c4","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b36c50c2891b887f03de5c4: start","job_status":"start","queue_duration_s":0.172996,"scheduling_latency_s":0.17301}2198gitlab # [ 259.483694] gitaly[895]: time="2026-09-18T22:14:12.062Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9B5S2TSQ6B5197QBYD9EA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-18T22:14:12.056" grpc.time_ms=5.805 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc2199gitlab # [ 259.495799] gitaly[895]: time="2026-09-18T22:14:12.133Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9B55J3JGC7NKV20D9KJG1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:12.133" grpc.time_ms=0.22 pid=895 span.kind=server system=grpc2200gitlab # [ 259.504914] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b36c50c2891b887f03de5c4","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b36c50c2891b887f03de5c4: done: 0.010404 sec","job_status":"done","queue_duration_s":0.172996,"scheduling_latency_s":0.17301,"redis_calls":13,"redis_duration_s":0.0018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001245,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010404,"completed_at":"2026-09-18T22:14:11.513Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201gitlab # [ 259.564164] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9ccc2200bf559b31f65c5a5","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9ccc2200bf559b31f65c5a5: start","job_status":"start","queue_duration_s":0.184304,"scheduling_latency_s":0.184318}2202gitlab # [ 259.564933] gitaly[895]: time="2026-09-18T22:14:12.154Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9B55J3JGC7NKV20D9KJG1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:12.153" grpc.time_ms=0.196 pid=895 span.kind=server system=grpc2203gitlab # [ 259.565652] gitaly[895]: time="2026-09-18T22:14:12.172Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2V9B55J3JGC7NKV20D9KJG1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-18T22:14:12.172" grpc.time_ms=0.358 pid=895 span.kind=server system=grpc2204gitlab # [ 259.569117] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9ccc2200bf559b31f65c5a5","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9ccc2200bf559b31f65c5a5: done: 0.010012 sec","job_status":"done","queue_duration_s":0.184304,"scheduling_latency_s":0.184318,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010012,"completed_at":"2026-09-18T22:14:11.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2205gitlab # [ 259.620688] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00656631699ccdd08b289e9b","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00656631699ccdd08b289e9b: start","job_status":"start","queue_duration_s":0.195267,"scheduling_latency_s":0.19528}2206gitlab # [ 259.621416] gitaly[895]: time="2026-09-18T22:14:12.181Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B55J3JGC7NKV20D9KJG1 diskcache=d06b3be0-632d-4f4d-aeba-dfb1f0b512b3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-18T22:14:42.013" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:12.113" pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2207gitlab # [ 259.624689] gitaly[895]: time="2026-09-18T22:14:12.181Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=67 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7502 command.oublock=0 command.real_time_ms=60 command.spawn_token_fork_ms=0 command.system_time_ms=44 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B55J3JGC7NKV20D9KJG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-18T22:14:42.013" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-18T22:14:12.113" grpc.time_ms=67.307 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2208gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses2209gitlab # [ 259.664508] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00656631699ccdd08b289e9b","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00656631699ccdd08b289e9b: done: 0.010054 sec","job_status":"done","queue_duration_s":0.195267,"scheduling_latency_s":0.19528,"redis_calls":13,"redis_duration_s":0.0016220000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010054,"completed_at":"2026-09-18T22:14:11.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2210gitlab # [ 259.672052] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"902e901d85958187159f8f20","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-902e901d85958187159f8f20: start","job_status":"start","queue_duration_s":0.206148,"scheduling_latency_s":0.206187}2211gitlab # [ 259.683653] gitaly[895]: time="2026-09-18T22:14:12.224Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2V9B55J3JGC7NKV20D9KJG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-18T22:14:42.123" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:12.224" grpc.time_ms=0.626 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2212gitlab # [ 259.728060] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"902e901d85958187159f8f20","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-902e901d85958187159f8f20: done: 0.009249 sec","job_status":"done","queue_duration_s":0.206148,"scheduling_latency_s":0.206187,"redis_calls":13,"redis_duration_s":0.00141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000447,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009249,"completed_at":"2026-09-18T22:14:11.545Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2213gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/789e09c2c59589392a122d7de0d22c17cad48131/statuses, in 0.08 seconds)2214(finished: subtest: eval and build post commit statuses, in 5.97 seconds)2215(finished: run the VM test script, in 322.41 seconds)2216gitlab # [ 259.729171] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57386a3bbce0fb262b7b892e","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c10d1ae8b016501a27245b152c5f4093","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-18T22:14:11.330Z","job_size_bytes":2,"pid":2539,"sidekiq_tid":"m7y7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57386a3bbce0fb262b7b892e: start","job_status":"start","queue_duration_s":0.216204,"scheduling_latency_s":0.216218}2217gitlab # [ 259.772059] gitaly[895]: time="2026-09-18T22:14:12.245Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2V9B55J3JGC7NKV20D9KJG1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-18T22:14:42.144" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-18T22:14:12.244" grpc.time_ms=0.515 pid=895 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2218test script finished in 322.43s2219cleanup2220kill QemuMachine (pid 46)2221gitlab # [ 259.772780] sidekiq-cluster[2539]: {"severity":"INFO","time":"2026-09-18T22:14:11.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57386a3bbce0fb262b7b892e","created_at":"2026-09-18T22:14:11.278Z","trace_propagation_headers":{"sentry-trace":"92c5dcc7be5b4fb899fd1d5b45dfc088-a94c03b147a54a6d","baggage":"sentry-trace_id=92c5dcc7be5b4fb899fd1d5b45dfc088,sentry-release=Unknown"},"meta.caller_id":"Ciqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2222gitlab # :2223(finished: cleanup, in 1.69 seconds)2224additionally exposed symbols:2225 gitlab,2226 vlan1,2227 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2228[]2229[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-18T22:13:14.781Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2230[]2231[]2232[]2233[]2234[]2235[]2236[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '789e09c2c59589392a122d7de0d22c17cad48131', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': [{'level': 'warning', 'message': "unable to download 'https://cache.nixos.org/…': Could not resolve hostname (6) Could not resolve host: cache.nixos.org", 'count': 1}], 'error': None, 'created_at': '2026-09-18T22:14:05.489407Z', 'started_at': '2026-09-18T22:14:05.949562Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2237[{'id': 5, 'sha': '789e09c2c59589392a122d7de0d22c17cad48131', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-18T22:14:05.922Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2238[{'id': 5, 'sha': '789e09c2c59589392a122d7de0d22c17cad48131', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-18T22:14:05.922Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2239[{'id': 5, 'sha': '789e09c2c59589392a122d7de0d22c17cad48131', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-18T22:14:05.922Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2240[{'id': 5, 'sha': '789e09c2c59589392a122d7de0d22c17cad48131', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-18T22:14:05.922Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2241[{'id': 5, 'sha': '789e09c2c59589392a122d7de0d22c17cad48131', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-18T22:14:05.922Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2242[{'id': 5, 'sha': '789e09c2c59589392a122d7de0d22c17cad48131', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-09-18T22:14:05.922Z', 'started_at': None, 'finished_at': '2026-09-18T22:14:11.485Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '789e09c2c59589392a122d7de0d22c17cad48131', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-18T22:14:11.586Z', 'started_at': None, 'finished_at': '2026-09-18T22:14:11.688Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]