vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #475
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.GtBfbcOom1', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: 003d9e24-ce7d-4654-8d06-d5f418616cdc16gitlab # Superblock backups stored on blocks:17(finished: start all VMs, in 51.13 seconds)18gitlab # 32768, 98304, 163840, 22937619gitlab # 20gitlab # Allocating group tables: 0/8 done21gitlab # Writing inode tables: 0/8 done22gitlab # Creating journal (8192 blocks): done23gitlab: waiting for unit setup-gitlab.service24gitlab: waiting for the VM to finish booting25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a46lfb5r7s7y0h1ynjz1aihwlhlhg7sa-nixos-system-gitlab-test/init regInfo=/nix/store/f7vnd2y17pym90xgdcy5cxkrcfbj3mkr-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 498038615 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000004] tsc: Detected 2400.008 MHz processor67gitlab # [ 0.000809] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000835] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000837] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000875] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002731] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002742] Using GB pages for direct mapping73gitlab # [ 0.002889] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002897] ACPI: Early table checksum verification disabled75gitlab # [ 0.002900] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002904] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002907] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002914] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002916] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002917] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002919] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002921] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002922] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002923] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002924] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002924] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002925] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002925] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003152] No NUMA configuration found90gitlab # [ 0.003153] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003155] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]92gitlab # [ 0.005477] Zone ranges:93gitlab # [ 0.005477] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005479] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005480] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005481] Device empty97gitlab # [ 0.005482] Movable zone start for each node98gitlab # [ 0.005482] Early memory node ranges99gitlab # [ 0.005483] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005484] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005485] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005486] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005505] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005774] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.116822] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.117237] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.117251] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.117281] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.117284] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.117286] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.117287] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.117288] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.117289] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.117291] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.117292] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.117297] TSC deadline timer available117gitlab # [ 0.117301] CPU topo: Max. logical packages: 1118gitlab # [ 0.117302] CPU topo: Max. logical dies: 1119gitlab # [ 0.117302] CPU topo: Max. dies per package: 1120gitlab # [ 0.117305] CPU topo: Max. threads per core: 1121gitlab # [ 0.117306] CPU topo: Num. cores per package: 4122gitlab # [ 0.117307] CPU topo: Num. threads per package: 4123gitlab # [ 0.117307] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.117331] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.117347] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.117350] kvm-guest: setup PV sched yield127gitlab # [ 0.117365] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.117366] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.117367] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.117369] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.117370] Booting paravirtualized kernel on KVM132gitlab # [ 0.117374] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.121816] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.124067] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.124116] kvm-guest: PV spinlocks enabled136gitlab # [ 0.124117] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.124120] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a46lfb5r7s7y0h1ynjz1aihwlhlhg7sa-nixos-system-gitlab-test/init regInfo=/nix/store/f7vnd2y17pym90xgdcy5cxkrcfbj3mkr-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.124215] Unknown kernel command line parameters "regInfo=/nix/store/f7vnd2y17pym90xgdcy5cxkrcfbj3mkr-closure-info/registration", will be passed to user space.139gitlab # [ 0.124228] random: crng init done140gitlab # [ 0.124228] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.133314] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.138068] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.138142] software IO TLB: area num 4.144gitlab # [ 0.214071] Fallback order for Node 0: 0145gitlab # [ 0.214079] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.214081] Policy zone: Normal147gitlab # [ 0.216650] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.226553] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.240440] allocated 12582912 bytes of page_ext150gitlab # [ 0.250366] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.250368] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.251261] Dynamic Preempt: lazy153gitlab # [ 0.251453] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.251454] rcu: RCU event tracing is enabled.155gitlab # [ 0.251454] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.251456] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.251457] Rude variant of Tasks RCU enabled.158gitlab # [ 0.251457] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.251458] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.251458] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.251480] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.251482] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.251483] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.255820] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.256113] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.256121] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.256227] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.259789] Console: colour VGA+ 80x25169gitlab # [ 0.259792] printk: legacy console [tty0] enabled170gitlab # [ 0.290490] printk: legacy console [ttyS0] enabled171gitlab # [ 0.400920] ACPI: Core revision 20250807172gitlab # [ 0.401923] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.403540] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.404621] x2apic enabled175gitlab # [ 0.405377] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.406330] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.407623] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.409128] kvm-guest: setup PV IPIs179gitlab # [ 0.410723] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.411804] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983f10a64, max_idle_ns: 440795218721 ns181gitlab # [ 0.413622] Calibrating delay loop (skipped) preset value.. 4800.01 BogoMIPS (lpj=2400008)182gitlab # [ 0.414706] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.415754] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.417620] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.418623] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.420620] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.421620] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.422620] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.423620] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.424620] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.425627] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.426620] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.427620] active return thunk: srso_alias_return_thunk194gitlab # [ 0.428641] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.429620] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.430620] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.431620] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.432620] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.434460] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.435506] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.436620] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.437620] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.438620] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.439620] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.440620] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.441620] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.472011] Freeing SMP alternatives memory: 44K208gitlab # [ 0.472624] pid_max: default: 32768 minimum: 301209gitlab # [ 0.473724] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.474725] landlock: Up and running.211gitlab # [ 0.475621] Yama: becoming mindful.212gitlab # [ 0.476486] LSM support for eBPF active213gitlab # [ 0.477362] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.478767] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.481589] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.482558] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.483512] ... version: 2218gitlab # [ 0.484260] ... bit width: 48219gitlab # [ 0.484673] ... generic counters: 6220gitlab # [ 0.485461] ... generic bitmap: 000000000000003f221gitlab # [ 0.486406] ... fixed-purpose counters: 0222gitlab # [ 0.486622] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.487561] ... value mask: 0000ffffffffffff224gitlab # [ 0.488353] ... max period: 00007fffffffffff225gitlab # [ 0.488622] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.489736] signal: max sigframe size: 3376227gitlab # [ 0.490555] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.491304] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.491817] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.497461] smp: Bringing up secondary CPUs ...231gitlab # [ 0.499157] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.499634] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.503684] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.505263] smpboot: Total of 4 processors activated (19200.06 BogoMIPS)235gitlab # [ 0.506229] Memory: 6011356K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved)236gitlab # [ 0.508624] devtmpfs: initialized237gitlab # [ 0.509858] x86/mm: Memory block size: 128MB238gitlab # [ 0.513679] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.514725] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.515740] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.516973] PM: RTC time: 18:44:24, date: 2026-10-25242gitlab # [ 0.521168] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.523261] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.523699] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.525922] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.526638] audit: initializing netlink subsys (disabled)247gitlab # [ 0.527802] audit: type=2000 audit(1792953864.046:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.527972] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.529624] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.530623] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.531623] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.532623] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.533640] cpuidle: using governor menu254gitlab # [ 0.535865] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.536877] PCI: Using configuration type 1 for base access256gitlab # [ 0.537624] PCI: Using configuration type 1 for extended access257gitlab # [ 0.538790] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.541838] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.542623] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.543623] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.544622] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.549800] ACPI: Added _OSI(Module Device)263gitlab # [ 0.551634] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.553630] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.560599] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.566291] ACPI: Interpreter enabled267gitlab # [ 0.567684] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.569632] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.571713] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.575632] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.577959] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.590647] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.592636] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.596271] acpiphp: Slot [3] registered275gitlab # [ 0.597648] acpiphp: Slot [4] registered276gitlab # [ 0.599667] acpiphp: Slot [5] registered277gitlab # [ 0.600700] acpiphp: Slot [6] registered278gitlab # [ 0.601666] acpiphp: Slot [7] registered279gitlab # [ 0.603666] acpiphp: Slot [8] registered280gitlab # [ 0.604682] acpiphp: Slot [9] registered281gitlab # [ 0.606668] acpiphp: Slot [10] registered282gitlab # [ 0.607666] acpiphp: Slot [11] registered283gitlab # [ 0.608666] acpiphp: Slot [12] registered284gitlab # [ 0.610654] acpiphp: Slot [13] registered285gitlab # [ 0.611427] acpiphp: Slot [14] registered286gitlab # [ 0.611642] acpiphp: Slot [15] registered287gitlab # [ 0.612415] acpiphp: Slot [16] registered288gitlab # [ 0.613660] acpiphp: Slot [17] registered289gitlab # [ 0.614461] acpiphp: Slot [18] registered290gitlab # [ 0.615179] acpiphp: Slot [19] registered291gitlab # [ 0.615642] acpiphp: Slot [20] registered292gitlab # [ 0.616420] acpiphp: Slot [21] registered293gitlab # [ 0.617644] acpiphp: Slot [22] registered294gitlab # [ 0.618387] acpiphp: Slot [23] registered295gitlab # [ 0.618651] acpiphp: Slot [24] registered296gitlab # [ 0.619436] acpiphp: Slot [25] registered297gitlab # [ 0.620643] acpiphp: Slot [26] registered298gitlab # [ 0.621415] acpiphp: Slot [27] registered299gitlab # [ 0.621668] acpiphp: Slot [28] registered300gitlab # [ 0.622398] acpiphp: Slot [29] registered301gitlab # [ 0.623645] acpiphp: Slot [30] registered302gitlab # [ 0.624414] acpiphp: Slot [31] registered303gitlab # [ 0.624634] PCI host bridge to bus 0000:00304gitlab # [ 0.625419] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.626557] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.627526] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.629622] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.630669] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.634655] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.637518] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.642070] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.646807] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.653711] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.655697] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.658640] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.660628] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.663628] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.667084] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.672697] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.677221] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.681161] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.683639] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.686061] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.704716] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.705637] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.706644] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.707972] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.710910] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.716658] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.717650] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.718645] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.731631] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.734208] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.738879] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.740629] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.741622] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.745242] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.749648] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.750570] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.753784] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.756053] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.760658] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.761600] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.762474] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.764981] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.770652] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.771570] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.772446] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.777712] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.790760] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.792649] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.795686] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.802203] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.811656] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.813645] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.815688] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.819646] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.824725] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.831653] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.833705] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.836888] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.843659] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.845647] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.847684] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.855121] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.863649] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.865648] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.867686] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.881097] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.884084] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.886996] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.888895] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.890788] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.895936] iommu: Default domain type: Translated375gitlab # [ 0.897627] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.900689] ACPI: bus type USB registered377gitlab # [ 0.901902] usbcore: registered new interface driver usbfs378gitlab # [ 0.904683] usbcore: registered new interface driver hub379gitlab # [ 0.907701] usbcore: registered new device driver usb380gitlab # [ 0.912969] NetLabel: Initializing381gitlab # [ 0.914634] NetLabel: domain hash size = 128382gitlab # [ 0.916633] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.919734] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.922662] PCI: Using ACPI for IRQ routing385gitlab # [ 0.925679] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.926619] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.926619] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.934642] vgaarb: loaded389gitlab # [ 0.936118] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.938626] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.944917] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.948366] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.949848] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.952928] pnp: PnP ACPI init395gitlab # [ 0.956010] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.970111] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.973866] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.977111] NET: Registered PF_INET protocol family399gitlab # [ 0.982297] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 1.011081] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 1.012659] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 1.014141] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 1.016828] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 1.018256] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 1.019967] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 1.021651] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 1.023789] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 1.028294] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 1.031340] NET: Registered PF_XDP protocol family410gitlab # [ 1.033767] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 1.036577] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 1.039377] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 1.042488] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 1.045615] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 1.049240] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 1.052027] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 1.059019] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 1.066208] pci 0000:00:01.2: quirk_usb_early_handoff+0x0/0x7c0 took 11007 usecs419gitlab # [ 1.070209] PCI: CLS 0 bytes, default 64420gitlab # [ 1.072369] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)421gitlab # [ 1.072981] Trying to unpack rootfs image as initramfs...422gitlab # [ 1.073039] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)423gitlab # [ 1.082152] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983f10a64, max_idle_ns: 440795218721 ns424gitlab # [ 1.135719] Initialise system trusted keyrings425gitlab # [ 1.137976] workingset: timestamp_bits=40 max_order=21 bucket_order=0426gitlab # [ 1.161489] Key type asymmetric registered427gitlab # [ 1.162314] Asymmetric key parser 'x509' registered428gitlab # [ 1.163290] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)429gitlab # [ 1.165066] io scheduler mq-deadline registered430gitlab # [ 1.165911] io scheduler kyber registered431gitlab # [ 1.169373] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled432gitlab # [ 1.170842] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A433gitlab # [ 1.173534] Linux agpgart interface v0.103434gitlab # [ 1.174381] ACPI: bus type drm_connector registered435gitlab # [ 1.177583] usbcore: registered new interface driver usbserial_generic436gitlab # [ 1.178757] usbserial: USB Serial support registered for generic437gitlab # [ 1.179851] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.438gitlab # [ 1.179851] Please enable it if your BIOS has the CPPC option.439gitlab # [ 1.182388] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled440gitlab # [ 1.183933] drop_monitor: Initializing network drop monitor service441gitlab # [ 1.185253] NET: Registered PF_INET6 protocol family442gitlab # [ 1.187164] Segment Routing with IPv6443gitlab # [ 1.187906] In-situ OAM (IOAM) with IPv6444gitlab # [ 1.190329] IPI shorthand broadcast: enabled445gitlab # [ 1.194905] sched_clock: Marking stable (1046015164, 148498650)->(1226161831, -31648017)446gitlab # [ 1.196850] registered taskstats version 1447gitlab # [ 1.197933] Loading compiled-in X.509 certificates448gitlab # [ 1.208057] Demotion targets for Node 0: null449gitlab # [ 1.209267] Key type .fscrypt registered450gitlab # [ 1.210065] Key type fscrypt-provisioning registered451gitlab # [ 1.211264] ima: No TPM chip found, activating TPM-bypass!452gitlab # [ 1.212337] ima: Allocated hash algorithm: sha1453gitlab # [ 1.213296] ima: No architecture policies found454gitlab # [ 1.214365] PM: Magic number: 14:498:747455gitlab # [ 1.216077] RAS: Correctable Errors collector initialized.456gitlab # [ 1.221019] clk: Disabling unused clocks457gitlab # [ 1.221827] PM: genpd: Disabling unused power domains458gitlab # [ 1.366373] Freeing initrd memory: 29392K459gitlab # [ 1.373907] Freeing unused decrypted memory: 2028K460gitlab # [ 1.380423] Freeing unused kernel image (initmem) memory: 3640K461gitlab # [ 1.383300] Write protecting the kernel read-only data: 32768k462gitlab # [ 1.388220] Freeing unused kernel image (text/rodata gap) memory: 1220K463gitlab # [ 1.392191] Freeing unused kernel image (rodata/data gap) memory: 760K464gitlab # [ 1.459311] x86/mm: Checked W+X mappings: passed, no W+X pages found.465gitlab # [ 1.460527] Run /init as init process466gitlab # [ 1.472432] systemd[1]: Inserted module 'autofs4'467gitlab # [ 1.494188] fuse: init (API version 7.45)468gitlab # [ 1.502207] ACPI: \_SB_.LNKC: Enabled at IRQ 10469gitlab # [ 1.509010] ACPI: \_SB_.LNKA: Enabled at IRQ 10470gitlab # [ 1.512822] ACPI: \_SB_.LNKB: Enabled at IRQ 11471gitlab # [ 1.547938] systemd[1]: Successfully made /usr/ read-only.472gitlab # [ 1.884776] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)473gitlab # [ 1.926849] systemd[1]: Detected virtualization kvm.474gitlab # [ 1.929086] systemd[1]: Detected architecture x86-64.475gitlab # [ 1.931330] systemd[1]: Running in initrd.476gitlab # [ 1.933887] systemd[1]: Initializing machine ID from random generator.477gitlab # [ 1.936706] systemd[1]: Hostname set to <gitlab>.478gitlab # [ 2.045727] systemd[1]: bpf-restrict-fs: LSM BPF program attached479gitlab # [ 2.088760] systemd[1]: Queued start job for default target Initrd Default Target.480gitlab # [ 2.100179] systemd[1]: Created slice Slice /system/modprobe.481gitlab # [ 2.101417] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.482gitlab # [ 2.102942] systemd[1]: Expecting device /dev/disk/by-label/nix-store...483gitlab # [ 2.104149] systemd[1]: Expecting device /dev/disk/by-label/nixos...484gitlab # [ 2.105342] systemd[1]: Reached target Path Units.485gitlab # [ 2.106285] systemd[1]: Reached target Slice Units.486gitlab # [ 2.107232] systemd[1]: Reached target Swaps.487gitlab # [ 2.108107] systemd[1]: Reached target Timer Units.488gitlab # [ 2.109138] systemd[1]: Listening on D-Bus System Message Bus Socket.489gitlab # [ 2.110411] systemd[1]: Listening on Journal Socket (/dev/log).490gitlab # [ 2.111658] systemd[1]: Listening on Journal Sockets.491gitlab # [ 2.112717] systemd[1]: Listening on udev Control Socket.492gitlab # [ 2.113779] systemd[1]: Listening on udev Kernel Socket.493gitlab # [ 2.114760] systemd[1]: Reached target Socket Units.494gitlab # [ 2.116733] systemd[1]: Starting Create List of Static Device Nodes...495gitlab # [ 2.118895] systemd[1]: Starting Load Kernel Module 9pnet_virtio...496gitlab # [ 2.121085] systemd[1]: Starting Load Kernel Module configfs...497gitlab # [ 2.126585] systemd[1]: Starting Journal Service...498gitlab # [ 2.128785] systemd[1]: Starting Load Kernel Modules...499gitlab # [ 2.129784] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os500gitlab # [ 2.133641] systemd[1]: Starting Coldplug All udev Devices...501gitlab # [ 2.146604] systemd[1]: Finished Create List of Static Device Nodes.502gitlab # [ 2.151765] systemd[1]: modprobe@configfs.service: Deactivated successfully.503gitlab # [ 2.155888] systemd[1]: Finished Load Kernel Module configfs.504gitlab # [ 2.157228] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config505gitlab # [ 2.165719] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...506gitlab # [ 2.173411] systemd-journald[91]: Collecting audit messages is disabled.507gitlab # [ 2.177196] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.508gitlab # [ 2.180074] netfs: FS-Cache loaded509gitlab # [ 2.186793] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev510gitlab # [ 2.187627] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.511gitlab # [ 2.192982] systemd[1]: Starting Create Static Device Nodes in /dev...512gitlab # [ 2.196415] 9pnet: Installing 9P2000 support513gitlab # [ 2.201974] systemd[1]: Finished Load Kernel Modules.514gitlab # [ 2.203321] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.515gitlab # [ 2.205048] systemd[1]: Finished Load Kernel Module 9pnet_virtio.516gitlab # [ 2.207320] systemd[1]: Starting Apply Kernel Variables...517gitlab # [ 2.209692] systemd[1]: Finished Create Static Device Nodes in /dev.518gitlab # [ 2.211087] systemd[1]: Reached target Preparation for Local File Systems.519gitlab # [ 2.212330] systemd[1]: Reached target Local File Systems.520gitlab # [ 2.214449] systemd[1]: Starting Rule-based Manager for Device Events and Files...521gitlab # [ 2.219107] systemd[1]: Finished Apply Kernel Variables.522gitlab # [ 2.071083] systemd-modules-l[ 2.220280] systemd[1]: Started Journal Service.523gitlab # oad[94]: Using 4 probe threads524gitlab # [ 2.073373] systemd-modules-load[94]: Inserted module 'virtio_balloon'525gitlab # [ 2.074477] systemd-modules-load[94]: Inserted module 'dm_mod'526gitlab # [ 2.075501] systemd-modules-load[94]: Inserted module 'virtio_gpu'527gitlab # [ 2.076630] systemd[1]: Starting Create System Files and Directories...528gitlab # [ 2.082859] systemd[1]: Finished Create System Files and Directories.529gitlab # [ 2.087698] systemd-udevd[105]: Using default interface naming scheme 'v261'.530gitlab # [ 2.098129] systemd[1]: Started Rule-based Manager for Device Events and Files.531gitlab # [ 2.127426] systemd[1]: Finished Coldplug All udev Devices.532gitlab # [ 2.128577] systemd[1]: Reached target System Initialization.533gitlab # [ 2.129631] systemd[1]: Reached target Basic System.534gitlab # [ 2.343745] virtio_blk virtio4: 4/0/0 default/read/poll queues535gitlab # [ 2.359638] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)536gitlab # [ 2.367831] uhci_hcd 0000:00:01.2: UHCI Host Controller537gitlab # [ 2.368344] SCSI subsystem initialized538gitlab # [ 2.368591] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1539gitlab # [ 2.370208] uhci_hcd 0000:00:01.2: detected 2 ports540gitlab # [ 2.371027] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140541gitlab # [ 2.372089] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18542gitlab # [ 2.373375] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1543gitlab # [ 2.374461] usb usb1: Product: UHCI Host Controller544gitlab # [ 2.375220] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd545gitlab # [ 2.376051] usb usb1: SerialNumber: 0000:00:01.2546gitlab # [ 2.376382] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12547gitlab # [ 2.380791] hub 1-0:1.0: USB hub found548gitlab # [ 2.381903] hub 1-0:1.0: 2 ports detected549gitlab # [ 2.385429] serio: i8042 KBD port at 0x60,0x64 irq 1550gitlab # [ 2.386222] serio: i8042 AUX port at 0x60,0x64 irq 12551gitlab # [ 2.407949] virtio_blk virtio5: 4/0/0 default/read/poll queues552gitlab # [ 2.411032] virtio_blk virtio5: [vdb] 11953312 512-byte logical blocks (6.12 GB/5.70 GiB)553gitlab # [ 2.428900] scsi host0: ata_piix554gitlab # [ 2.432890] scsi host1: ata_piix555gitlab # [ 2.285275] systemd[1]: Starting Virtual Console Setup...556gitlab # [ 2.286753] (udev-worker)[127]: Network interface NamePolicy= disabled on kernel command line.557gitlab # [ 2.437455] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0558gitlab # [ 2.440203] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0559gitlab # [ 2.441327] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0560gitlab # [ 2.296529] (udev-worker)[133]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.561gitlab # [ 2.298697] (udev-worker)[133]: Network interface NamePolicy= disabled on kernel command line.562gitlab # [ 2.310524] systemd-vconsole-setup[149]: Configuration of first virtual console was skipped, ignoring remaining ones.563gitlab # [ 2.313815] systemd[1]: Found device /dev/disk/by-label/nixos.564gitlab # [ 2.315316] systemd[1]: Finished Virtual Console Setup.565gitlab # [ 2.318453] systemd[1]: Found device /dev/disk/by-label/nix-store.566gitlab # [ 2.319376] systemd[1]: Reached target Initrd Root Device.567gitlab # [ 2.320154] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...568gitlab # [ 2.347145] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks569gitlab # [ 2.350297] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.570gitlab # [ 2.606144] ata2: found unknown device (class 0)571gitlab # [ 2.608978] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100572gitlab # [ 2.613295] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5573gitlab # [ 2.613683] usb 1-1: new full-speed USB device number 2 using uhci_hcd574gitlab # [ 2.665918] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray575gitlab # [ 2.693327] cdrom: Uniform CD-ROM driver Revision: 3.20576gitlab # [ 2.785064] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00577gitlab # [ 2.788111] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10578gitlab # [ 2.790816] usb 1-1: Product: QEMU USB Tablet579gitlab # [ 2.792470] usb 1-1: Manufacturer: QEMU580gitlab # [ 2.794052] usb 1-1: SerialNumber: 28754-0000:00:01.2-1581gitlab # [ 2.826090] hid: raw HID events driver (C) Jiri Kosina582gitlab # [ 2.840276] usbcore: registered new interface driver usbhid583gitlab # [ 2.842450] usbhid: USB HID core driver584gitlab # [ 2.852577] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2585gitlab # [ 2.856243] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0586gitlab # [ 2.992540] systemd[1]: Mounting /sysroot...587gitlab # [ 3.296144] EXT4-fs (vda): mounted filesystem 003d9e24-ce7d-4654-8d06-d5f418616cdc r/w with ordered data mode. Quota mode: none.588gitlab # [ 3.154720] systemd[1]: Mounted /sysroot.589gitlab # [ 3.156334] systemd[1]: Reached target Initrd Root File System.590gitlab # [ 3.159911] systemd[1]: Mounting /sysroot/nix/.ro-store...591gitlab # [ 3.161965] systemd[1]: Mounting /sysroot/nix/.rw-store...592gitlab # [ 3.165266] systemd[1]: Mounting /sysroot/run...593gitlab # [ 3.177187] systemd[1]: Mounting /sysroot/tmp/shared...594gitlab # [ 3.180517] systemd[1]: Mounting /sysroot/tmp/xchg...595gitlab # [ 3.186357] systemd[1]: Starting Mountpoints Configured in the Real Root...596gitlab # [ 3.193200] systemd[1]: Mounted /sysroot/nix/.rw-store.597gitlab # [ 3.197703] systemd-sysroot-fstab-check[176]: /sysroot should be mounted in the initrd, will request daemon-reload.598gitlab # [ 3.200622] systemd[1]: Mounted /sysroot/run.599gitlab # [ 3.354767] erofs (device vdb): mounted with root inode @ nid 128.600gitlab # [ 3.208359] systemd[1]: Starting rw-sysroot-nix-store.service...601gitlab # [ 3.210061] systemd[1]: Reload requested from client PID 176 ('systemd-sysroot') (unit initrd-parse-etc.service)...602gitlab # [ 3.215259] systemd[1]: Reloading...603gitlab # [ 3.364280] 9p: Installing v9fs 9p2000 file system support604gitlab # [ 3.268185] systemd[1]: Reloading finished in 63 ms.605gitlab # [ 3.295582] systemd-sysroot-fstab-check[176]: Requesting initrd-fs.target/start/replace...606gitlab # [ 3.298460] systemd[1]: Mounted /sysroot/nix/.ro-store.607gitlab # [ 3.300356] systemd[1]: Mounted /sysroot/tmp/shared.608gitlab # [ 3.302132] systemd[1]: Mounted /sysroot/tmp/xchg.609gitlab # [ 3.303797] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.610gitlab # [ 3.306350] systemd[1]: Finished rw-sysroot-nix-store.service.611gitlab # [ 3.308355] systemd-sysroot-fstab-check[176]: Requesting swap.target/start/replace...612gitlab # [ 3.310810] systemd[1]: Starting rw-sysroot-nix-store.service...613gitlab # [ 3.312812] systemd[1]: initrd-parse-etc.service: Deactivated successfully.614gitlab # [ 3.315115] systemd[1]: Finished Mountpoints Configured in the Real Root.615gitlab # [ 3.317279] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.616gitlab # [ 3.335505] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.617gitlab # [ 3.337874] systemd[1]: Finished rw-sysroot-nix-store.service.618gitlab # [ 3.992976] systemd[1]: Mounting /sysroot/nix/store...619gitlab # [ 4.043592] systemd[1]: Mounted /sysroot/nix/store.620gitlab # [ 4.046418] systemd[1]: Reached target Initrd File Systems.621gitlab # [ 4.050218] systemd[1]: Starting Find NixOS closure...622gitlab # [ 4.053578] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...623gitlab # [ 4.077296] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.624gitlab # [ 4.080330] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.625gitlab # [ 4.098255] systemd[1]: Finished Find NixOS closure.626gitlab # [ 4.100102] systemd[1]: Reached target Initrd Default Target.627gitlab # [ 4.102545] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...628gitlab # [ 4.133123] systemd[1]: Stopped target Initrd Default Target.629gitlab # [ 4.135227] systemd[1]: Stopped target Basic System.630gitlab # [ 4.137115] systemd[1]: Stopped target Initrd Root Device.631gitlab # [ 4.138987] systemd[1]: Stopped target Path Units.632gitlab # [ 4.140840] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.633gitlab # [ 4.143658] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.634gitlab # [ 4.147324] systemd[1]: Stopped target Slice Units.635gitlab # [ 4.149265] systemd[1]: Stopped target Socket Units.636gitlab # [ 4.151217] systemd[1]: Stopped target System Initialization.637gitlab # [ 4.153329] systemd[1]: Stopped target Swaps.638gitlab # [ 4.155506] systemd[1]: Stopped target Timer Units.639gitlab # [ 4.157276] systemd[1]: dbus.socket: Deactivated successfully.640gitlab # [ 4.159457] systemd[1]: Closed D-Bus System Message Bus Socket.641gitlab # [ 4.161429] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.642gitlab # [ 4.163927] systemd[1]: Stopped Find NixOS closure.643gitlab # [ 4.165659] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio644gitlab # [ 4.169235] systemd[1]: Starting rw-sysroot-nix-store.service...645gitlab # [ 4.171170] systemd[1]: systemd-sysctl.service: Deactivated successfully.646gitlab # [ 4.173559] systemd[1]: Stopped Apply Kernel Variables.647gitlab # [ 4.175205] systemd[1]: systemd-modules-load.service: Deactivated successfully.648gitlab # [ 4.177511] systemd[1]: Stopped Load Kernel Modules.649gitlab # [ 4.179190] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.650gitlab # [ 4.181409] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.651gitlab # [ 4.183423] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.652gitlab # [ 4.185343] systemd[1]: Stopped Create System Files and Directories.653gitlab # [ 4.187066] systemd[1]: Stopped target Local File Systems.654gitlab # [ 4.188588] systemd[1]: Stopped target Preparation for Local File Systems.655gitlab # [ 4.190092] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.656gitlab # [ 4.191385] systemd[1]: Stopped Coldplug All udev Devices.657gitlab # [ 4.192444] systemd[1]: Stopping Rule-based Manager for Device Events and Files...658gitlab # [ 4.193776] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.659gitlab # [ 4.195136] systemd[1]: Stopped Virtual Console Setup.660gitlab # [ 4.196393] systemd[1]: initrd-cleanup.service: Deactivated successfully.661gitlab # [ 4.197636] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.662gitlab # [ 4.198812] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.663gitlab # [ 4.199978] systemd[1]: Finished rw-sysroot-nix-store.service.664gitlab # [ 4.200806] systemd[1]: systemd-udevd.service: Deactivated successfully.665gitlab # [ 4.201796] systemd[1]: Stopped Rule-based Manager for Device Events and Files.666gitlab # [ 4.202854] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.667gitlab # [ 4.203833] systemd[1]: Closed udev Control Socket.668gitlab # [ 4.204589] systemd[1]: Starting Cleanup udev Database...669gitlab # [ 4.205404] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.670gitlab # [ 4.206497] systemd[1]: Stopped Create Static Device Nodes in /dev.671gitlab # [ 4.207369] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.672gitlab # [ 4.208543] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.673gitlab # [ 4.209577] systemd[1]: kmod-static-nodes.service: Deactivated successfully.674gitlab # [ 4.210579] systemd[1]: Stopped Create List of Static Device Nodes.675gitlab # [ 4.211537] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.676gitlab # [ 4.212607] systemd[1]: Finished Cleanup udev Database.677gitlab # [ 4.213382] systemd[1]: Reached target Switch Root.678gitlab # [ 4.214099] systemd[1]: Starting NixOS Activation...679gitlab # [ 4.331802] initrd-nixos-activation-start[228]: booting system configuration /nix/store/a46lfb5r7s7y0h1ynjz1aihwlhlhg7sa-nixos-system-gitlab-test680gitlab # [ 4.371305] initrd-nixos-activation-start[228]: running activation script...681gitlab # [ 4.614605] initrd-nixos-activation-start[251]: setting up /etc...682gitlab # [ 4.729510] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.683gitlab # [ 4.731672] systemd[1]: Finished NixOS Activation.684gitlab # [ 4.732971] systemd[1]: Starting Switch Root...685gitlab # [ 4.770491] systemd[1]: Switching root.686gitlab # [ 4.984067] systemd-journald[91]: Received SIGTERM from PID 1 (systemd).687gitlab # [ 5.098367] NET: Registered PF_VSOCK protocol family688gitlab # [ 5.462066] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)689gitlab # [ 5.472244] systemd[1]: Detected virtualization kvm.690gitlab # [ 5.474180] systemd[1]: Detected architecture x86-64.691gitlab # [ 5.476227] systemd[1]: Detected first boot.692gitlab # [ 5.478874] systemd[1]: Initializing machine ID from random generator.693gitlab # [ 5.636551] systemd[1]: bpf-restrict-fs: LSM BPF program attached694gitlab # [ 5.720223] systemd[1]: Applying preset policy.695gitlab # [ 5.882351] systemd[1]: Populated /etc with preset unit settings.696gitlab # [ 5.998553] systemd[1]: initrd-switch-root.service: Deactivated successfully.697gitlab # [ 5.999964] systemd[1]: Stopped initrd-switch-root.service.698gitlab # [ 6.002686] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.699gitlab # [ 6.004832] systemd[1]: Created slice Slice /system/getty.700gitlab # [ 6.006289] systemd[1]: Created slice GitLab DevOps Platform Slice.701gitlab # [ 6.007858] systemd[1]: Created slice User and Session Slice.702gitlab # [ 6.008801] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.703gitlab # [ 6.010052] systemd[1]: Started Forward Password Requests to Wall Directory Watch.704gitlab # [ 6.011230] systemd[1]: Expecting device /dev/hvc0...705gitlab # [ 6.012045] systemd[1]: Expecting device /dev/ttyS0...706gitlab # [ 6.012848] systemd[1]: Reached target Local Encrypted Volumes.707gitlab # [ 6.013792] systemd[1]: Stopped target initrd-fs.target.708gitlab # [ 6.014557] systemd[1]: Stopped target initrd-root-fs.target.709gitlab # [ 6.015368] systemd[1]: Stopped target initrd-switch-root.target.710gitlab # [ 6.016287] systemd[1]: Reached target Virtual Machines and Containers.711gitlab # [ 6.017322] systemd[1]: Reached target Path Units.712gitlab # [ 6.018111] systemd[1]: Reached target Remote File Systems.713gitlab # [ 6.019004] systemd[1]: Reached target Slice Units.714gitlab # [ 6.019776] systemd[1]: Reached target Swaps.715gitlab # [ 6.021159] systemd[1]: Listening on Query the User Interactively for a Password.716gitlab # [ 6.023167] systemd[1]: Listening on Process Core Dump Socket.717gitlab # [ 6.024653] systemd[1]: Listening on Credential Encryption/Decryption.718gitlab # [ 6.026243] systemd[1]: Listening on Factory Reset Management.719gitlab # [ 6.027222] systemd[1]: Listening on Hostname Service Socket.720gitlab # [ 6.029645] systemd[1]: Starting Journal Log Access Socket...721gitlab # [ 6.031092] systemd[1]: Listening on Journal Audit Socket.722gitlab # [ 6.033043] systemd[1]: Listening on Console Output Muting Service Socket.723gitlab # [ 6.034252] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.724gitlab # [ 6.035445] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os725gitlab # [ 6.036869] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki726gitlab # [ 6.039701] systemd[1]: Listening on Disk Repartitioning Service Socket.727gitlab # [ 6.040822] systemd[1]: Listening on udev Control Socket.728gitlab # [ 6.041755] systemd[1]: Listening on udev Varlink Socket.729gitlab # [ 6.044845] systemd[1]: Mounting Huge Pages File System...730gitlab # [ 6.046861] systemd[1]: Mounting POSIX Message Queue File System...731gitlab # [ 6.048999] systemd[1]: Mounting Kernel Debug File System...732gitlab # [ 6.053135] systemd[1]: Mounting Kernel Trace File System...733gitlab # [ 6.064019] systemd[1]: Starting Create List of Static Device Nodes...734gitlab # [ 6.064981] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio735gitlab # [ 6.084575] systemd[1]: Starting Load Kernel Module configfs...736gitlab # [ 6.085587] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm737gitlab # [ 6.087103] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore738gitlab # [ 6.088789] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse739gitlab # [ 6.095329] systemd[1]: Mounting FUSE Control File System...740gitlab # [ 6.096312] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67741gitlab # [ 6.100438] systemd[1]: Starting Journal Service...742gitlab # [ 6.103636] systemd[1]: Starting Load Kernel Modules...743gitlab # [ 6.114925] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...744gitlab # [ 6.120570] systemd[1]: Starting Remount Root and Kernel File Systems...745gitlab # [ 6.121565] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os746gitlab # [ 6.125713] systemd[1]: Starting Coldplug All udev Devices...747gitlab # [ 6.128637] systemd[1]: Listening on Journal Log Access Socket.748gitlab # [ 6.129689] systemd[1]: Mounted Huge Pages File System.749gitlab # [ 6.130668] systemd[1]: Mounted POSIX Message Queue File System.750gitlab # [ 6.131709] systemd[1]: Mounted Kernel Debug File System.751gitlab # [ 6.132636] systemd[1]: Mounted Kernel Trace File System.752gitlab # [ 6.135813] systemd[1]: Finished Create List of Static Device Nodes.753gitlab # [ 6.137408] systemd-journald[322]: Collecting audit messages is enabled.754gitlab # [ 6.137862] systemd[1]: modprobe@configfs.service: Deactivated successfully.755gitlab # [ 6.140036] systemd[1]: Finished Load Kernel Module configfs.756gitlab # [ 6.141205] systemd[1]: Mounted FUSE Control File System.757gitlab # [ 6.144551] loop: module loaded758gitlab # [ 6.146424] systemd[1]: Mounting Kernel Configuration File System...759gitlab # [ 6.148673] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...760gitlab # [ 6.152281] systemd[1]: Finished Load Kernel Modules.761gitlab # [ 6.153250] EXT4-fs (vda): re-mounted 003d9e24-ce7d-4654-8d06-d5f418616cdc.762gitlab # [ 6.008891] systemd[1]: Queued start job for default target Multi-User System.763gitlab # [ 6.010083] systemd[1]: Unnecessary job was removed for /dev/vdb.764gitlab # [ 6.011164] systemd[1]: systemd-journald.service: Deactivated successfully.765gitlab # [ 6.012416] systemd-modules-load[323]: Using 3 probe threads766gitlab # [ 6.013942] systemd-modules-load[323]: Inserted module 'loop'767gitlab # [ 6.163356] systemd[1]: Starting Firewall...768gitlab # [ 6.015908] systemd-modules-load[323]: Inserted module 'tls'769gitlab # [ 6.168630] systemd[1]: Starting Apply Kernel Variables...770gitlab # [ 6.170389] systemd[1]: Started Journal Service.771gitlab # [ 6.025453] systemd[1]: Finished Remount Root and Kernel File Systems.772gitlab # [ 6.026444] systemd[1]: Mounted Kernel Configuration File System.773gitlab # [ 6.027815] systemd-oomd[324]: No swap; memory pressure usage will be degraded774gitlab # [ 6.030312] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.775gitlab # [ 6.031522] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.776gitlab # [ 6.033968] systemd[1]: Listening on Disk Image Download Service Socket.777gitlab # [ 6.035169] systemd[1]: Starting Flush Journal to Persistent Storage...778gitlab # [ 6.036325] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore779gitlab # [ 6.038076] systemd[1]: Starting Load/Save OS Random Seed...780gitlab # [ 6.039173] systemd[1]: Starting Create Static Device Nodes in /dev...781gitlab # [ 6.040079] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os782gitlab # [ 6.050838] systemd[1]: Finished Apply Kernel Variables.783gitlab # [ 6.203222] systemd-journald[322]: Received client request to flush runtime journal.784gitlab # [ 6.099116] systemd[1]: Finished Create Static Device Nodes in /dev.785gitlab # [ 6.100482] systemd[1]: Reached target Preparation for Local File Systems.786gitlab # [ 6.101840] systemd[1]: Starting Rule-based Manager for Device Events and Files...787gitlab # [ 6.103154] systemd[1]: Finished Load/Save OS Random Seed.788gitlab # [ 6.104100] systemd[1]: Reached target First Boot Complete.789gitlab # [ 6.105214] systemd-udevd[359]: Using default interface naming scheme 'v261'.790gitlab # [ 6.106306] systemd[1]: Finished Flush Journal to Persistent Storage.791gitlab # [ 6.129199] systemd[1]: Started Rule-based Manager for Device Events and Files.792gitlab # [ 6.141644] systemd[1]: Finished Coldplug All udev Devices.793gitlab # [ 6.200482] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse794gitlab # [ 6.221798] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.795gitlab # [ 6.246562] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.796gitlab # [ 6.251844] (udev-worker)[440]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.797gitlab # [ 6.254078] (udev-worker)[440]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 6.263755] systemd[1]: Condition check resulted in Virtio network device being skipped.799gitlab # [ 6.266520] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore800gitlab # [ 6.268218] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67801gitlab # [ 6.270456] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore802gitlab # [ 6.272098] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os803gitlab # [ 6.273360] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os804gitlab # [ 6.280265] systemd[1]: Finished Firewall.805gitlab # [ 6.281607] (udev-worker)[419]: Network interface NamePolicy= disabled on kernel command line.806gitlab # [ 6.455812] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3807gitlab # [ 6.462974] ACPI: button: Power Button [PWRF]808gitlab # [ 6.467556] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console809gitlab # [ 6.476617] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0810gitlab # [ 6.476678] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD811gitlab # [ 6.480168] Console: switching to colour dummy device 80x25812gitlab # [ 6.482525] [drm] Found bochs VGA, ID 0xb0c5.813gitlab # [ 6.483033] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.814gitlab # [ 6.488242] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic815gitlab # [ 6.489085] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0816gitlab # [ 6.492738] rtc_cmos 00:05: RTC can wake from S4817gitlab # [ 6.494697] Floppy drive(s): fd0 is 2.88M AMI BIOS818gitlab # [ 6.499265] parport_pc 00:03: reported by Plug and Play ACPI819gitlab # [ 6.504173] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]820gitlab # [ 6.505837] FDC 0 is a S82078B821gitlab # [ 6.506444] Console: switching to colour frame buffer device 160x50822gitlab # [ 6.508990] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device823gitlab # [ 6.511985] rtc_cmos 00:05: registered as rtc0824gitlab # [ 6.512694] rtc_cmos 00:05: setting system clock to 2026-10-25T18:44:30 UTC (1792953870)825gitlab # [ 6.513888] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs826gitlab # [ 6.546455] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5827gitlab # [ 6.551254] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4828gitlab # [ 6.587138] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6829gitlab # [ 6.600401] ppdev: user-space parallel port driver830gitlab # [ 6.461065] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 6.623992] mousedev: PS/2 mouse device common for all mice832gitlab # [ 6.490577] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.833gitlab # [ 6.494159] systemd[1]: Stopped Virtual Console Setup.834gitlab # [ 6.505294] systemd[1]: Starting Virtual Console Setup...835gitlab # [ 6.518107] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.836gitlab # [ 6.519704] systemd[1]: Stopped Virtual Console Setup.837gitlab # [ 6.521233] systemd[1]: Starting Virtual Console Setup...838gitlab # [ 6.700603] kvm_amd: TSC scaling supported839gitlab # [ 6.701111] kvm_amd: Nested Virtualization enabled840gitlab # [ 6.701781] kvm_amd: Nested Paging enabled841gitlab # [ 6.702278] kvm_amd: LBR virtualization supported842gitlab # [ 6.702910] kvm_amd: Virtual VMLOAD VMSAVE supported843gitlab # [ 6.703575] kvm_amd: Virtual GIF supported844gitlab # [ 6.704073] kvm_amd: Virtual NMI enabled845gitlab # [ 6.577666] systemd-vconsole-setup[498]: Configuration of first virtual console was skipped, ignoring remaining ones.846gitlab # [ 6.580147] systemd[1]: Finished Virtual Console Setup.847gitlab # [ 6.740877] EDAC MC: Ver: 3.0.0848gitlab # [ 6.855234] systemd[1]: Mounting /run/wrappers...849gitlab # [ 6.877859] systemd[1]: Mounted /run/wrappers.850gitlab # [ 6.880503] systemd[1]: Reached target Local File Systems.851gitlab # [ 6.883659] systemd[1]: Listening on Boot Loader Control Service Socket.852gitlab # [ 6.888751] systemd[1]: Starting register-nix-paths.service...853gitlab # [ 6.892888] systemd[1]: Starting Create SUID/SGID Wrappers...854gitlab # [ 6.894862] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.855gitlab # [ 6.903699] systemd[1]: Starting Save Transient machine-id to Disk...856gitlab # [ 6.908316] systemd[1]: Starting Create System Files and Directories...857gitlab # [ 6.940112] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.858gitlab # [ 6.944883] systemd[1]: Finished Save Transient machine-id to Disk.859gitlab # [ 6.956077] systemd[1]: Finished Create System Files and Directories.860gitlab # [ 6.959814] systemd[1]: Starting Rebuild Journal Catalog...861gitlab # [ 6.962500] systemd[1]: Starting Record System Boot/Shutdown in UTMP...862gitlab # [ 6.980679] systemd[1]: Finished Record System Boot/Shutdown in UTMP.863gitlab # [ 6.990194] systemd[1]: Finished Rebuild Journal Catalog.864gitlab # [ 6.992689] systemd[1]: Starting Update is Completed...865gitlab # [ 7.003425] systemd[1]: Finished Update is Completed.866gitlab # [ 7.039697] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.867gitlab # [ 7.041441] systemd[1]: Finished Create SUID/SGID Wrappers.868gitlab # [ 7.267904] systemd[1]: Finished register-nix-paths.service.869gitlab # [ 7.270309] systemd[1]: Reached target System Initialization.870gitlab # [ 7.272404] systemd[1]: Started Discard unused filesystem blocks once a week.871gitlab # [ 7.274887] systemd[1]: Started logrotate.timer.872gitlab # [ 7.276567] systemd[1]: Started Daily Cleanup of Temporary Directories.873gitlab # [ 7.278854] systemd[1]: Reached target Timer Units.874gitlab # [ 7.280689] systemd[1]: Listening on D-Bus System Message Bus Socket.875gitlab # [ 7.282963] systemd[1]: Listening on Nix Daemon Socket.876gitlab # [ 7.284946] systemd[1]: Starting nixbot CI service socket...877gitlab # [ 7.287063] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.878gitlab # [ 7.289988] systemd[1]: Starting D-Bus System Message Bus...879gitlab # [ 7.292216] systemd[1]: Listening on nixbot CI service socket.880gitlab # [ 7.294651] systemd[1]: Reached target Socket Units.881gitlab # [ 7.337630] dbus-broker-launch[581]: Looking up NSS user entry for 'systemd-timesync'...882gitlab # [ 7.340781] dbus-broker-launch[581]: NSS returned no entry for 'systemd-timesync'883gitlab # [ 7.343568] dbus-broker-launch[581]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"884gitlab # [ 7.348975] systemd[1]: Started D-Bus System Message Bus.885gitlab # [ 7.364297] dbus-broker-launch[581]: Ready886gitlab # [ 7.369067] systemd[1]: Reached target Basic System.887gitlab # [ 7.372287] systemd[1]: Started backdoor.service.888gitlab # [ 7.377309] systemd[1]: Starting gitlab-config.service...889gitlab # [ 7.379450] systemd[1]: Starting Import lastlog data into lastlog2 database...890gitlab # [ 7.394159] systemd[1]: Starting Logrotate configuration check...891gitlab # [ 7.401507] systemd[1]: Starting Name Service Cache Daemon (nsncd)...892gitlab # [ 7.408475] systemd[1]: Starting Post-Boot Actions...893gitlab # [ 7.417269] logrotate[586]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.894gitlab # [ 7.424400] logrotate[586]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf895gitlab # [ 7.428211] logrotate[586]: note: 'monthly' overrides previously specified 'weekly'896gitlab # connecting to host...897gitlab # [ 7.433611] logrotate[586]: Reading state from file: /var/lib/logrotate.status898gitlab # [ 7.435722] logrotate[586]: state file /var/lib/logrotate.status does not exist899gitlab # [ 7.437788] logrotate[586]: Allocating hash table for state file, size 64 entries900gitlab # [ 7.439120] logrotate[586]: Handling 4 logs901gitlab # [ 7.439660] logrotate[586]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed902gitlab: Guest shell says: b'Spawning backdoor root shell...\n'903gitlab # [ 7.440680] logrotate[586]: considering log /var/log/btmp904gitlab # [ 7.442054] logrotate[586]: Creating new state905gitlab: connected to guest root shell906gitlab # [ 7.442673] logrotate[586]: Now: 2026-10-25 18:44907gitlab: (connecting took 8.08 seconds)908gitlab # [ 7.444111] logrotate[586]: Last rotated at 2026-10-25 18:00909gitlab: (finished: waiting for the VM to finish booting, in 8.08 seconds)910gitlab # [ 7.444653] logrotate[586]: log does not need rotating (log has already been rotated)911gitlab # [ 7.445954] logrotate[586]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed912gitlab # [ 7.446881] logrotate[586]: considering log /var/log/wtmp913gitlab # [ 7.447743] logrotate[586]: Creating new state914gitlab # [ 7.448650] logrotate[586]: Now: 2026-10-25 18:44915gitlab # [ 7.448951] logrotate[586]: Last rotated at 2026-10-25 18:00916gitlab # [ 7.449836] logrotate[586]: log does not need rotating (log has already been rotated)917gitlab # [ 7.450447] logrotate[586]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed918gitlab # [ 7.451057] logrotate[586]: switching euid from 0 to 165 and egid from 0 to 165 (pid 586)919gitlab # [ 7.451654] logrotate[586]: considering log /var/gitlab/state/log/*.log920gitlab # [ 7.452812] logrotate[586]: log /var/gitlab/state/log/*.log does not exist -- skipping921gitlab # [ 7.453713] logrotate[586]: Creating new state922gitlab # [ 7.454196] logrotate[586]: switching euid from 165 to 0 and egid from 165 to 0 (pid 586)923gitlab # [ 7.454677] logrotate[586]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed924gitlab # [ 7.455126] logrotate[586]: switching euid from 0 to 60 and egid from 0 to 60 (pid 586)925gitlab # [ 7.455592] logrotate[586]: considering log /var/log/nginx/*.log926gitlab # [ 7.456050] logrotate[586]: log /var/log/nginx/*.log does not exist -- skipping927gitlab # [ 7.456513] logrotate[586]: Creating new state928gitlab # [ 7.457267] logrotate[586]: not running postrotate script, since no logs were rotated929gitlab # [ 7.457750] logrotate[586]: switching euid from 60 to 0 and egid from 60 to 0 (pid 586)930gitlab # [ 7.458450] systemd[1]: Started Reset console on configuration changes.931gitlab # [ 7.459192] nsncd[587]: Oct 25 18:44:31.579 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"932gitlab # [ 7.459942] systemd[1]: Starting resolvconf update...933gitlab # [ 7.460759] systemd[1]: Finished Logrotate configuration check.934gitlab # [ 7.461374] systemd[1]: Finished Post-Boot Actions.935gitlab # [ 7.474682] systemd[1]: Started Name Service Cache Daemon (nsncd).936gitlab # [ 7.476813] systemd[1]: Finished Import lastlog data into lastlog2 database.937gitlab # [ 7.478910] systemd[1]: Reached target Host and Network Name Lookups.938gitlab # [ 7.479830] systemd[1]: Reached target User and Group Name Lookups.939gitlab # [ 7.483566] systemd[1]: Starting User Login Management...940gitlab # [ 7.526274] systemd[1]: Stopped target Host and Network Name Lookups.941gitlab # [ 7.527024] systemd[1]: Stopping Host and Network Name Lookups...942gitlab # [ 7.527442] systemd[1]: Stopped target User and Group Name Lookups.943gitlab # [ 7.527995] systemd[1]: Stopping User and Group Name Lookups...944gitlab # [ 7.528496] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...945gitlab # [ 7.531168] systemd[1]: nscd.service: Deactivated successfully.946gitlab # [ 7.532116] systemd-logind[620]: New seat seat0.947gitlab # [ 7.533125] systemd[1]: Stopped Name Service Cache Daemon (nsncd).948gitlab # [ 7.533976] systemd[1]: nscd.service: Consumed 22ms CPU time over 129ms wall clock time, 7.5M memory peak, 5.5M read from disk.949gitlab # [ 7.537144] systemd-logind[620]: Watching system buttons on /dev/input/event2 (Power Button)950gitlab # [ 7.538478] systemd-logind[620]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)951gitlab # [ 7.540310] systemd-logind[620]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)952gitlab # [ 7.541832] systemd[1]: Started User Login Management.953gitlab # [ 7.542656] systemd[1]: Starting linger-users.service...954gitlab # [ 7.543409] systemd[1]: Starting Name Service Cache Daemon (nsncd)...955gitlab # [ 7.557451] systemd[1]: linger-users.service: Deactivated successfully.956gitlab # [ 7.558405] systemd[1]: Finished linger-users.service.957gitlab # [ 7.561088] nsncd[680]: Oct 25 18:44:31.695 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"958gitlab # [ 7.561772] systemd[1]: Started Name Service Cache Daemon (nsncd).959gitlab # [ 7.562246] systemd[1]: Reached target Host and Network Name Lookups.960gitlab # [ 7.562704] systemd[1]: Reached target User and Group Name Lookups.961gitlab # [ 7.566067] systemd[1]: Finished resolvconf update.962gitlab # [ 7.566802] systemd[1]: Reached target Preparation for Network.963gitlab # [ 7.568941] systemd[1]: Starting DHCP Client...964gitlab # [ 7.571315] systemd[1]: Starting Address configuration of eth1...965gitlab # [ 7.574074] systemd[1]: Starting Extra networking commands....966gitlab # [ 7.602215] network-addresses-eth1-start[704]: adding address 192.168.1.1/24... done967gitlab # [ 7.604952] network-addresses-eth1-start[704]: adding address 2001:db8:1::1/64... done968gitlab # [ 7.610460] systemd[1]: Finished Address configuration of eth1.969gitlab # [ 7.636041] dhcpcd[711]: dhcpcd-10.3.2 starting970gitlab # [ 7.638776] dhcpcd[768]: dev: loaded udev971gitlab # [ 7.640756] systemd[1]: Finished Extra networking commands..972gitlab # [ 7.641674] systemd[1]: Reached target Network.973gitlab # [ 7.642499] systemd[1]: Starting gitlab-workhorse.service...974gitlab # [ 7.793818] 8021q: 802.1Q VLAN Support v1.8975gitlab # [ 7.794306] 8021q: adding VLAN 0 to HW filter on device eth1976gitlab # [ 7.646223] systemd[1]: Starting Nginx Web Server...977gitlab # [ 7.650336] systemd[1]: Starting PostgreSQL Server...978gitlab # [ 7.655107] systemd[1]: Starting Redis Server - redis-gitlab...979gitlab # [ 7.656745] systemd[1]: Starting Permit User Sessions...980gitlab # [ 7.664615] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.981gitlab # [ 7.679350] systemd[1]: Finished Permit User Sessions.982gitlab # [ 7.683754] systemd[1]: Started Getty on tty1.983gitlab # [ 7.685672] systemd[1]: Reached target Login Prompts.984gitlab # [ 7.881517] cfg80211: Loading compiled-in X.509 certificates for regulatory database985gitlab # [ 7.885255] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'986gitlab # [ 7.886176] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'987gitlab # [ 7.887836] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2988gitlab # [ 7.889473] cfg80211: failed to load regulatory.db989gitlab # [ 7.914172] 8021q: adding VLAN 0 to HW filter on device eth0990gitlab # [ 7.767450] dhcpcd[768]: eth0: waiting for carrier991gitlab # [ 7.768285] dhcpcd[768]: eth0: carrier acquired992gitlab # [ 7.773643] dhcpcd[768]: DUID 00:01:00:01:32:71:0a:8f:52:54:00:12:34:56993gitlab # [ 7.774701] dhcpcd[768]: eth0: IAID 00:12:34:56994gitlab # [ 7.775570] dhcpcd[768]: eth0: adding address fe80::5054:ff:fe12:3456995gitlab # [ 7.777899] nginx-pre-start[789]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok996gitlab # [ 7.778475] nginx-pre-start[789]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful997gitlab # [ 7.784108] systemd[1]: Started Nginx Web Server.998gitlab # [ 7.797683] redis-server[794]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.999gitlab # [ 7.799478] redis-server[794]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1000gitlab # [ 7.800787] redis-server[794]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=794, just started1001gitlab # [ 7.801784] redis-server[794]: Configuration loaded1002gitlab # [ 7.802904] redis[794]: monotonic clock: POSIX clock_gettime1003gitlab # [ 7.803741] redis[794]: Running mode=standalone, port=0.1004gitlab # [ 7.805351] redis[794]: Server initialized1005gitlab # [ 7.806612] postgresql-pre-start[802]: The files belonging to this database system will be owned by user "postgres".1006gitlab # [ 7.808415] postgresql-pre-start[802]: This user must also own the server process.1007gitlab # [ 7.809817] redis[794]: Ready to accept connections unix1008gitlab # [ 7.810938] systemd[1]: Started Redis Server - redis-gitlab.1009gitlab # [ 7.811907] postgresql-pre-start[802]: The database cluster will be initialized with locale "en_US.UTF-8".1010gitlab # [ 7.813318] postgresql-pre-start[802]: The default database encoding has accordingly been set to "UTF8".1011gitlab # [ 7.814854] postgresql-pre-start[802]: The default text search configuration will be set to "english".1012gitlab # [ 7.816117] postgresql-pre-start[802]: Data page checksums are enabled.1013gitlab # [ 7.817191] postgresql-pre-start[802]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1014gitlab # [ 7.818661] postgresql-pre-start[802]: creating subdirectories ... ok1015gitlab # [ 7.819576] postgresql-pre-start[802]: selecting dynamic shared memory implementation ... posix1016gitlab # [ 7.820860] redis[794]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1017gitlab # [ 7.843231] postgresql-pre-start[802]: selecting default "max_connections" ... 1001018gitlab # [ 7.887920] postgresql-pre-start[802]: selecting default "shared_buffers" ... 128MB1019gitlab # [ 8.162130] systemd[1]: Started gitlab-workhorse.service.1020gitlab # [ 8.290095] postgresql-pre-start[802]: selecting default time zone ... UTC1021gitlab # [ 8.291518] postgresql-pre-start[802]: creating configuration files ... ok1022gitlab # [ 8.354417] gitlab-workhorse[829]: time="2026-10-25T18:44:32Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41023gitlab # [ 8.355129] gitlab-workhorse[829]: time="2026-10-25T18:44:32Z" level=info msg="Using redis/go-redis"1024gitlab # [ 8.355566] gitlab-workhorse[829]: time="2026-10-25T18:44:32Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1025gitlab # [ 8.449431] postgresql-pre-start[802]: running bootstrap script ... ok1026gitlab # [ 8.869456] postgresql-pre-start[802]: performing post-bootstrap initialization ... ok1027gitlab # [ 9.020525] dhcpcd[768]: eth0: soliciting an IPv6 router1028gitlab # [ 9.022216] dhcpcd[768]: eth0: Router Advertisement from fe80::21029gitlab # [ 9.024234] dhcpcd[768]: eth0: adding address fec0::5054:ff:fe12:3456/641030gitlab # [ 9.026185] dhcpcd[768]: eth0: adding route to fec0::/641031gitlab # [ 9.028180] dhcpcd[768]: eth0: adding default route via fe80::21032gitlab # [ 9.206164] postgresql-pre-start[802]: syncing data to disk ... ok1033gitlab # [ 9.208144] postgresql-pre-start[802]: initdb: warning: enabling "trust" authentication for local connections1034gitlab # [ 9.209509] postgresql-pre-start[802]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1035gitlab # [ 9.211087] postgresql-pre-start[802]: Success. You can now start the database server using:1036gitlab # [ 9.212214] postgresql-pre-start[802]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1037gitlab # [ 9.266354] dhcpcd[768]: eth0: soliciting a DHCP lease1038gitlab # [ 9.421211] NET: Registered PF_PACKET protocol family1039gitlab # [ 9.277553] dhcpcd[768]: eth0: offered 10.0.2.15 from 10.0.2.21040gitlab # [ 9.286538] dhcpcd[768]: eth0: probing address 10.0.2.15/241041gitlab # [ 9.355159] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1042gitlab # [ 9.356542] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 54321043gitlab # [ 9.357908] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 54321044gitlab # [ 9.367521] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1045gitlab # [ 9.383492] postgres[865]: [865] LOG: database system was shut down at 2026-10-25 18:44:32 GMT1046gitlab # [ 9.393195] postgres[854]: [854] LOG: database system is ready to accept connections1047gitlab # [ 9.400552] systemd[1]: Started PostgreSQL Server.1048gitlab # [ 9.407196] systemd[1]: Starting PostgreSQL Setup Scripts...1049gitlab # [ 9.554744] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[873]: mkdir -p /var/gitlab/state/home/.ssh: OK1050gitlab # [ 9.560802] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[873]: chmod 700 /var/gitlab/state/home/.ssh: OK1051gitlab # [ 9.610515] postgresql-setup-start[881]: CREATE DATABASE1052gitlab # [ 9.610898] systemd[1]: Finished gitlab-config.service.1053gitlab # [ 9.612912] systemd[1]: Started gitaly.service.1054gitlab # [ 9.635518] postgresql-setup-start[909]: CREATE ROLE1055gitlab # [ 9.644770] postgresql-setup-start[911]: ALTER DATABASE1056gitlab # [ 9.661317] postgresql-setup-start[916]: CREATE ROLE1057gitlab # [ 9.662857] systemd[1]: Finished PostgreSQL Setup Scripts.1058gitlab # [ 9.663957] systemd[1]: Reached target PostgreSQL.1059gitlab # [ 9.666089] systemd[1]: Starting gitlab-db-config.service...1060gitlab # [ 9.668589] systemd[1]: Starting gitlab-postgresql.service...1061gitlab # [ 9.755955] gitlab-postgresql-start[926]: CREATE DATABASE1062gitlab # [ 9.790916] gitlab-postgresql-start[936]: CREATE EXTENSION1063gitlab # [ 9.866983] gitlab-postgresql-start[938]: CREATE EXTENSION1064gitlab # [ 9.869614] systemd[1]: Finished gitlab-postgresql.service.1065gitlab # [ 10.106990] gitaly[904]: time="2026-10-25T18:44:34.242Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9041066gitlab # [ 10.108368] gitaly[904]: time="2026-10-25T18:44:34.242Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9041067gitlab # [ 10.112628] gitaly[904]: time="2026-10-25T18:44:34.242Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9041068gitlab # [ 10.113623] gitaly[904]: time="2026-10-25T18:44:34.242Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9041069gitlab # [ 10.114488] gitaly[904]: time="2026-10-25T18:44:34.242Z" level=info msg="Starting Gitaly" pid=904 version=1070gitlab # [ 10.279693] gitaly[904]: time="2026-10-25T18:44:34.415Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=172 pid=9041071gitlab # [ 10.280592] gitaly[904]: time="2026-10-25T18:44:34.415Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9041072gitlab # [ 10.281026] gitaly[904]: time="2026-10-25T18:44:34.415Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9041073gitlab # [ 10.281636] gitaly[904]: time="2026-10-25T18:44:34.415Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-904/git-exec-2448728849.d/git pid=9041074gitlab # [ 10.283611] gitaly[904]: time="2026-10-25T18:44:34.419Z" level=info msg="finished detecting git version" duration_ms=3 pid=9041075gitlab # [ 10.284656] gitaly[904]: time="2026-10-25T18:44:34.419Z" level=info msg="using Git version" pid=904 version=2.53.g1076gitlab # [ 10.285287] gitaly[904]: time="2026-10-25T18:44:34.419Z" level=info msg="clearing disk cache object folder" pid=904 storage=default1077gitlab # [ 10.285685] gitaly[904]: time="2026-10-25T18:44:34.419Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3244375495 pid=904 storage=default1078gitlab # [ 10.286491] gitaly[904]: time="2026-10-25T18:44:34.419Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3244375495 pid=904 storage=default1079gitlab # [ 10.287162] gitaly[904]: time="2026-10-25T18:44:34.419Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9041080gitlab # [ 10.287765] gitaly[904]: time="2026-10-25T18:44:34.420Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9041081gitlab # [ 10.288606] gitaly[904]: time="2026-10-25T18:44:34.420Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3244375495 clear_duration_ms=0 pid=904 storage=default1082gitlab # [ 10.338039] gitaly[904]: time="2026-10-25T18:44:34.473Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9041083gitlab # [ 10.338764] gitaly[904]: time="2026-10-25T18:44:34.473Z" level=info msg="counting repositories" pid=9041084gitlab # [ 10.339216] gitaly[904]: time="2026-10-25T18:44:34.473Z" level=info msg="starting to count repositories" pid=904 storage_path=/var/gitlab/state/repositories1085gitlab # [ 10.339711] gitaly[904]: time="2026-10-25T18:44:34.473Z" level=info msg="finished tempdir cleaner walk" pid=904 storage=default time_ms=01086gitlab # [ 10.340230] gitaly[904]: time="2026-10-25T18:44:34.473Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=904 schema=unix1087gitlab # [ 10.340693] gitaly[904]: time="2026-10-25T18:44:34.473Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=904 storage_path=/var/gitlab/state/repositories1088gitlab # [ 10.341149] gitaly[904]: time="2026-10-25T18:44:34.473Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9041089gitlab # [ 10.341683] gitaly[904]: time="2026-10-25T18:44:34.473Z" level=info msg="listening at address" address=/run/gitaly/gitaly-904/sock.d/intern pid=904 schema=unix1090gitlab # [ 10.351567] gitaly[904]: time="2026-10-25T18:44:34.487Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9041091gitlab # [ 10.352190] gitaly[904]: time="2026-10-25T18:44:34.487Z" level=info msg="Started Gitaly" duration_ms=244 pid=9041092gitlab # [ 10.352662] gitaly[904]: time="2026-10-25T18:44:34.487Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9041093gitlab # [ 10.353132] gitaly[904]: time="2026-10-25T18:44:34.487Z" level=info msg="maintenance: daily scheduled" pid=904 scheduled="2026-10-26 12:00:00 +0000 UTC"1094gitlab # [ 11.115109] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[921]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1095gitlab # [ 11.117857] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[921]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1096gitlab # [ 11.142863] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[921]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1097gitlab # [ 12.265523] gitaly[904]: time="2026-10-25T18:44:36.400Z" level=info msg="License database preloaded" duration_ms=2158 pid=9041098gitlab # [ 14.766185] dhcpcd[768]: eth0: leased 10.0.2.15 for 86400 seconds1099gitlab # [ 14.768419] dhcpcd[768]: eth0: adding route to 10.0.2.0/241100gitlab # [ 14.770380] dhcpcd[768]: eth0: adding default route via 10.0.2.21101gitlab # [ 14.894546] systemd[1]: Started DHCP Client.1102gitlab # [ 14.897675] systemd[1]: Reached target Network is Online.1103gitlab # [ 40.943942] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[921]: W, [2026-10-25T18:45:05.078947 #921] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1104gitlab # [ 43.039429] postgres[1311]: [1311] ERROR: relation "feature_gates" does not exist at character 5231105gitlab # [ 43.042438] postgres[1311]: [1311] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1106gitlab # [ 43.045384] postgres[1311]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1107gitlab # [ 43.048490] postgres[1311]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1108gitlab # [ 43.051352] postgres[1311]: attidentity AS identity,1109gitlab # [ 43.053319] postgres[1311]: attgenerated as attgenerated1110gitlab # [ 43.055383] postgres[1311]: FROM pg_attribute a1111gitlab # [ 43.057099] postgres[1311]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1112gitlab # [ 43.060138] postgres[1311]: LEFT JOIN pg_type t ON a.atttypid = t.oid1113gitlab # [ 43.062380] postgres[1311]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1114gitlab # [ 43.065663] postgres[1311]: WHERE a.attrelid = '"feature_gates"'::regclass1115gitlab # [ 43.067972] postgres[1311]: AND a.attnum > 0 AND NOT a.attisdropped1116gitlab # [ 43.070170] postgres[1311]: ORDER BY a.attnum1117gitlab # [ 43.071830] postgres[1311]:1118gitlab # [ 46.305845] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1353]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1119gitlab # [ 46.307139] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1353]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1120gitlab # [ 48.400205] gitlab-workhorse[829]: time="2026-10-25T18:45:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1121gitlab # [ 58.400940] gitlab-workhorse[829]: time="2026-10-25T18:45:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 61.128324] postgres[1419]: [1419] ERROR: relation "table_without_models" does not exist1123gitlab # [ 61.130601] postgres[1419]: [1419] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1124gitlab # [ 68.410537] gitlab-workhorse[829]: time="2026-10-25T18:45:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1125gitlab # [ 72.823298] systemd[1]: Finished gitlab-db-config.service.1126gitlab # [ 72.828190] systemd[1]: Started gitlab-sidekiq.service.1127gitlab # [ 72.829812] systemd[1]: Starting gitlab.service...1128gitlab # [ 73.160585] bundle[1586]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1129gitlab # [ 73.161778] bundle[1586]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1130gitlab # [ 73.165387] bundle[1586]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1131gitlab # [ 75.851954] sidekiq-cluster[1585]: {"severity":"INFO","time":"2026-10-25T18:45:39.970Z","message":"Starting cluster with 1 processes"}1132gitlab # [ 75.903322] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.018Z","pid":1586,"message":"* Enabling systemd notification integration"}1133gitlab # [ 75.904401] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"Puma starting in cluster mode..."}1134gitlab # [ 75.905173] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1135gitlab # [ 75.906092] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1136gitlab # [ 75.906868] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"* Min threads: 0"}1137gitlab # [ 75.907663] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"* Max threads: 4"}1138gitlab # [ 75.908306] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"* Environment: production"}1139gitlab # [ 75.908953] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"* Master PID: 1586"}1140gitlab # [ 75.909658] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"* Workers: 2"}1141gitlab # [ 75.910478] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1142gitlab # [ 75.911163] bundle[1586]: {"timestamp":"2026-10-25T18:45:40.038Z","pid":1586,"message":"* Preloading application"}1143gitlab # [ 78.419648] gitlab-workhorse[829]: time="2026-10-25T18:45:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1144gitlab # [ 88.420771] gitlab-workhorse[829]: time="2026-10-25T18:45:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 97.773324] bundle[1586]: W, [2026-10-25T18:46:01.908137 #1586] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1146gitlab # [ 98.431331] gitlab-workhorse[829]: time="2026-10-25T18:46:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1147gitlab # [ 100.464417] sidekiq-cluster[1616]: 2026-10-25T18:46:04.599Z pid=1616 tid=wl4 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1616, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430530560}1148gitlab # [ 100.633927] sidekiq-cluster[1616]: W, [2026-10-25T18:46:04.769610 #1616] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1149gitlab # [ 100.674694] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:04.810Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1150gitlab # [ 102.491404] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.626Z","message":"GitLab reliable fetch activated!"}1151gitlab # [ 102.492598] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.626Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1152gitlab # [ 102.517190] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.652Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1153gitlab # [ 102.521578] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.657Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1154gitlab # [ 102.526305] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.661Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1155gitlab # [ 102.530856] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.666Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1156gitlab # [ 102.535703] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.671Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1157gitlab # [ 102.540332] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.675Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1158gitlab # [ 102.544752] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.680Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1159gitlab # [ 102.548840] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.684Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1160gitlab # [ 102.553203] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.688Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1161gitlab # [ 102.557194] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.692Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1162gitlab # [ 102.561767] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.697Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1163gitlab # [ 102.568975] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.704Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1164gitlab # [ 102.574353] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.709Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1165gitlab # [ 102.579684] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.715Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1166gitlab # [ 102.585109] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.720Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1167gitlab # [ 102.590666] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.726Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1168gitlab # [ 102.597369] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.732Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1169gitlab # [ 102.604100] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.739Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1170gitlab # [ 102.610864] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.746Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1171gitlab # [ 102.617352] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.752Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1172gitlab # [ 102.624101] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.759Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1173gitlab # [ 102.630343] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.765Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1174gitlab # [ 102.636349] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.771Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1175gitlab # [ 102.641732] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.777Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1176gitlab # [ 102.647348] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.782Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1177gitlab # [ 102.654228] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.789Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1178gitlab # [ 102.660649] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.796Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1179gitlab # [ 102.666565] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.802Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1180gitlab # [ 102.671964] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.807Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1181gitlab # [ 102.678400] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.814Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1182gitlab # [ 102.683533] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.819Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1183gitlab # [ 102.691223] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.826Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1184gitlab # [ 102.695880] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.831Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1185gitlab # [ 102.700349] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.835Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1186gitlab # [ 102.706140] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.841Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1187gitlab # [ 102.709631] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.845Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1188gitlab # [ 102.713309] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.848Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1189gitlab # [ 102.738947] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.873Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1190gitlab # [ 102.743327] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.878Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1191gitlab # [ 102.748482] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.884Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1192gitlab # [ 102.753808] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.889Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1193gitlab # [ 102.759462] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.894Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1194gitlab # [ 102.765396] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.900Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1195gitlab # [ 102.768916] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.904Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1196gitlab # [ 102.771257] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.906Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1197gitlab # [ 102.773433] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.908Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1198gitlab # [ 102.775761] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.910Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1199gitlab # [ 102.777727] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.912Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1200gitlab # [ 102.779719] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.914Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1201gitlab # [ 102.781711] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.915Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1202gitlab # [ 102.783669] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.918Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1203gitlab # [ 102.785727] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.920Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1204gitlab # [ 102.793959] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.929Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1205gitlab # [ 102.796520] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.932Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1206gitlab # [ 102.800495] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.936Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1207gitlab # [ 102.804576] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.940Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1208gitlab # [ 102.808562] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.944Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1209gitlab # [ 102.815976] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.951Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1210gitlab # [ 102.820848] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.956Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1211gitlab # [ 102.826721] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.962Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1212gitlab # [ 102.831836] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.967Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1213gitlab # [ 102.836553] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.972Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1214gitlab # [ 102.842729] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.978Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1215gitlab # [ 102.848949] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.984Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1216gitlab # [ 102.854653] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:06.990Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1217gitlab # [ 102.892911] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.027Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1218gitlab # [ 102.901084] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.036Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1219gitlab # [ 102.909212] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.044Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1220gitlab # [ 102.917634] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.053Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1221gitlab # [ 102.925477] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.061Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1222gitlab # [ 102.933534] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.069Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1223gitlab # [ 102.940688] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.076Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1224gitlab # [ 102.948546] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.084Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1225gitlab # [ 102.956193] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.091Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1226gitlab # [ 102.964215] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.099Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1227gitlab # [ 102.971871] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.107Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1228gitlab # [ 102.979966] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.115Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1229gitlab # [ 102.987804] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.123Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1230gitlab # [ 102.994927] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.130Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1231gitlab # [ 103.001827] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.137Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1232gitlab # [ 103.009438] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.145Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1233gitlab # [ 103.016691] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.152Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1234gitlab # [ 103.024977] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.160Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1235gitlab # [ 103.031718] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.167Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1236gitlab # [ 103.037081] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.172Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1237gitlab # [ 103.043108] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.178Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1238gitlab # [ 103.049307] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.184Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1239gitlab # [ 103.055335] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.190Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1240gitlab # [ 103.060561] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.196Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1241gitlab # [ 103.067092] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.202Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1242gitlab # [ 103.073107] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.208Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1243gitlab # [ 103.079253] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.214Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1244gitlab # [ 103.084503] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.220Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1245gitlab # [ 103.089996] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.225Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1246gitlab # [ 103.097122] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.232Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1247gitlab # [ 103.104339] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.239Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1248gitlab # [ 103.109791] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.245Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1249gitlab # [ 103.115714] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.251Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1250gitlab # [ 103.135281] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.270Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1251gitlab # [ 103.140115] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.275Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1252gitlab # [ 103.145402] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.281Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1253gitlab # [ 103.151751] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.287Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1254gitlab # [ 103.158582] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.294Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1255gitlab # [ 103.164104] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.299Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1256gitlab # [ 103.170465] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.306Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1257gitlab # [ 103.175208] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.310Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1258gitlab # [ 103.180504] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.316Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1259gitlab # [ 103.187078] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.322Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1260gitlab # [ 103.193425] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:07.328Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1261gitlab # [ 105.658348] sidekiq-cluster[1616]: InitializerConnections Query: SET client_min_messages TO 'warning'1262gitlab # [ 105.660699] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1263gitlab # [ 105.661996] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1264gitlab # [ 105.663444] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1265gitlab # [ 105.665137] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1266gitlab # [ 105.666539] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1267gitlab # [ 105.668233] sidekiq-cluster[1616]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1268gitlab # [ 105.669655] sidekiq-cluster[1616]: InitializerConnections Query: SET standard_conforming_strings = on1269gitlab # [ 105.671267] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1270gitlab # [ 105.672100] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1271gitlab # [ 105.672859] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1272gitlab # [ 105.674973] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1273gitlab # [ 105.677119] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1274gitlab # [ 105.677558] sidekiq-cluster[1616]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1275gitlab # [ 105.703454] sidekiq-cluster[1616]: InitializerConnections Query: SET intervalstyle = iso_86011276gitlab # [ 105.705076] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1277gitlab # [ 105.706367] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1278gitlab # [ 105.707414] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1279gitlab # [ 105.708409] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1280gitlab # [ 105.709371] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1281gitlab # [ 105.710368] sidekiq-cluster[1616]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1282gitlab # [ 105.722913] sidekiq-cluster[1616]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname1283gitlab # [ 105.723966] sidekiq-cluster[1616]: FROM pg_type as t1284gitlab # [ 105.724775] sidekiq-cluster[1616]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')1285gitlab # [ 105.725539] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1286gitlab # [ 105.726303] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1287gitlab # [ 105.727077] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1288gitlab # [ 105.727835] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1289gitlab # [ 105.728481] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1290gitlab # [ 105.729212] sidekiq-cluster[1616]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1291gitlab # [ 105.742480] sidekiq-cluster[1616]: InitializerConnections Query: SET SESSION timezone TO 'UTC'1292gitlab # [ 105.743664] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1293gitlab # [ 105.744543] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1294gitlab # [ 105.745510] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1295gitlab # [ 105.746259] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1296gitlab # [ 105.746982] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1297gitlab # [ 105.747732] sidekiq-cluster[1616]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1298gitlab # [ 105.756821] sidekiq-cluster[1616]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics'1299gitlab # [ 105.760232] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1300gitlab # [ 105.762099] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1301gitlab # [ 105.763150] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1302gitlab # [ 105.763874] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1303gitlab # [ 105.765073] sidekiq-cluster[1616]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1304gitlab # [ 105.765781] sidekiq-cluster[1616]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1305gitlab # [ 108.441219] gitlab-workhorse[829]: time="2026-10-25T18:46:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1306gitlab # [ 118.450910] gitlab-workhorse[829]: time="2026-10-25T18:46:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1307gitlab # [ 127.034985] postgres[2054]: [2054] ERROR: relation "table_without_models" does not exist1308gitlab # [ 127.036217] postgres[2054]: [2054] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1309gitlab # [ 127.391869] postgres[2055]: [2055] ERROR: relation "table_without_models" does not exist1310gitlab # [ 127.394699] postgres[2055]: [2055] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1311gitlab # [ 128.452967] gitlab-workhorse[829]: time="2026-10-25T18:46:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1312gitlab # [ 129.583149] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:33.715Z","message":"Booted Rails 7.2.3.1 application in production environment"}1313gitlab # [ 129.587649] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:33.718Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1314gitlab # [ 129.592351] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:33.718Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1315gitlab # [ 129.596515] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:33.718Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1316gitlab # [ 129.601306] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:33.719Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1317gitlab # [ 130.116272] bundle[1586]: {"timestamp":"2026-10-25T18:46:34.251Z","pid":1586,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1318gitlab # [ 130.120574] bundle[1586]: {"timestamp":"2026-10-25T18:46:34.251Z","pid":1586,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1319gitlab # [ 130.124611] bundle[1586]: {"timestamp":"2026-10-25T18:46:34.251Z","pid":1586,"message":"! #\u003cThread:0x00007a4933f5fe28 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1320gitlab # [ 130.136213] bundle[1586]: {"timestamp":"2026-10-25T18:46:34.251Z","pid":1586,"message":"! #\u003cThread:0x00007a49341e6a20 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1321gitlab # [ 130.147689] bundle[1586]: {"timestamp":"2026-10-25T18:46:34.251Z","pid":1586,"message":"Use Ctrl-C to stop"}1322gitlab # [ 130.150701] bundle[1586]: {"timestamp":"2026-10-25T18:46:34.251Z","pid":1586,"message":"Sending status to systemd every 1.0 sec"}1323gitlab # [ 130.456202] bundle[1586]: {"timestamp":"2026-10-25T18:46:34.591Z","pid":1586,"message":"- Worker 0 (PID: 2150) booted in 0.31s, phase: 0"}1324gitlab # [ 130.489869] bundle[1586]: {"timestamp":"2026-10-25T18:46:34.625Z","pid":1586,"message":"- Worker 1 (PID: 2154) booted in 0.32s, phase: 0"}1325gitlab # [ 130.494394] systemd[1]: Started gitlab.service.1326gitlab # [ 130.496270] systemd[1]: Reached target Common target for all GitLab services..1327gitlab # [ 130.498740] systemd[1]: Starting setup-gitlab.service...1328gitlab # [ 130.517663] setup-gitlab-start[2178]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1329gitlab # [ 130.836925] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:34.971Z","message":"Cleaning working queues"}1330gitlab # [ 130.840589] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:34.973Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5b2ac452827d945115a16212","created_at":"2026-10-25T18:45:36.639Z","trace_propagation_headers":{"sentry-trace":"eff2cd17d82740e792680ceed2dec44d-17dcfa1ac5284fe5","baggage":"sentry-trace_id=eff2cd17d82740e792680ceed2dec44d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"1737bc65004d8ab1ed25052f06a4bd85","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-25T18:45:36.646Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5b2ac452827d945115a16212: start","job_status":"start","queue_duration_s":58.326684,"scheduling_latency_s":58.326724}1331gitlab # [ 130.990298] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:46:35.125Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5b2ac452827d945115a16212","created_at":"2026-10-25T18:45:36.639Z","trace_propagation_headers":{"sentry-trace":"eff2cd17d82740e792680ceed2dec44d-17dcfa1ac5284fe5","baggage":"sentry-trace_id=eff2cd17d82740e792680ceed2dec44d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"1737bc65004d8ab1ed25052f06a4bd85","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-25T18:45:36.646Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5b2ac452827d945115a16212: done: 0.152641 sec","job_status":"done","queue_duration_s":58.326684,"scheduling_latency_s":58.326724,"redis_calls":26,"redis_duration_s":0.0057799999999999995,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003309,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001447,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08825954100000001,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.152641,"completed_at":"2026-10-25T18:46:35.125Z","load_balancing_strategy":"primary","db_duration_s":0.065799,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1332gitlab # [ 132.733334] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:46:36 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 21881333gitlab # [ 132.738106] setup-gitlab-start[2178]: + TOKEN=nixbot-test-token-of-20-chars1334gitlab # [ 132.740704] setup-gitlab-start[2178]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1335gitlab # [ 132.752751] runuser[2203]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1336gitlab # [ 133.094203] setup-gitlab-start[2204]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1337gitlab # [ 133.099263] setup-gitlab-start[2204]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1338gitlab # [ 133.104099] setup-gitlab-start[2204]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1339gitlab # [ 133.661354] setup-gitlab-start[2204]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1340gitlab # [ 133.666312] setup-gitlab-start[2204]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1341gitlab # [ 133.671051] setup-gitlab-start[2204]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1342gitlab # [ 158.332774] setup-gitlab-start[2204]: W, [2026-10-25T18:47:02.467628 #2204] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1343gitlab # [ 161.451815] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:05.586Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2e14c684dd1a916b20033713","created_at":"2026-10-25T18:47:05.519Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c53710468a25fe12abe400c1ef718206","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.583Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2e14c684dd1a916b20033713: start","job_status":"start","queue_duration_s":0.002424,"scheduling_latency_s":0.00247}1344gitlab # [ 161.948936] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.084Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2e14c684dd1a916b20033713","created_at":"2026-10-25T18:47:05.519Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c53710468a25fe12abe400c1ef718206","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.583Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2e14c684dd1a916b20033713: done: 0.498055 sec","job_status":"done","queue_duration_s":0.002424,"scheduling_latency_s":0.00247,"redis_calls":14,"redis_duration_s":0.004674,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001482,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001374,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.074,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.07,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.225154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001176407,"exclusive_lock_hold_duration_s":0.391849522,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.498055,"completed_at":"2026-10-25T18:47:06.084Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.202904,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 162.017726] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.153Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b8145dfdd5f96e5f23197196","created_at":"2026-10-25T18:47:05.597Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be1e6fce35df7b4c6066f670623acdd8","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.619Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b8145dfdd5f96e5f23197196: start","job_status":"start","queue_duration_s":0.533588,"scheduling_latency_s":0.533653}1346gitlab # [ 162.051773] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.186Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b8145dfdd5f96e5f23197196","created_at":"2026-10-25T18:47:05.597Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be1e6fce35df7b4c6066f670623acdd8","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.619Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b8145dfdd5f96e5f23197196: done: 0.033804 sec","job_status":"done","queue_duration_s":0.533588,"scheduling_latency_s":0.533653,"redis_calls":12,"redis_duration_s":0.004031,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00073,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001819,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001482,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000784178,"exclusive_lock_hold_duration_s":0.004769321,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.033804,"completed_at":"2026-10-25T18:47:06.186Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 162.072757] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.189Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e52100ad446e1ae6932f302c","created_at":"2026-10-25T18:47:05.630Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90065e39422da56ec489d01c64c756ad","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.637Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e52100ad446e1ae6932f302c: start","job_status":"start","queue_duration_s":0.551398,"scheduling_latency_s":0.551426}1348gitlab # [ 162.106860] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.242Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e52100ad446e1ae6932f302c","created_at":"2026-10-25T18:47:05.630Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90065e39422da56ec489d01c64c756ad","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.637Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e52100ad446e1ae6932f302c: done: 0.053082 sec","job_status":"done","queue_duration_s":0.551398,"scheduling_latency_s":0.551426,"redis_calls":16,"redis_duration_s":0.004679,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00241,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000567,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043106,"duration_s":0.053082,"completed_at":"2026-10-25T18:47:06.242Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 162.144653] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.243Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"65d30bd9d48c68718085de5f","created_at":"2026-10-25T18:47:05.671Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58411b4f3f851c0abc88000910c50c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.681Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-65d30bd9d48c68718085de5f: start","job_status":"start","queue_duration_s":0.562763,"scheduling_latency_s":0.562787}1350gitlab # [ 162.145621] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.267Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"65d30bd9d48c68718085de5f","created_at":"2026-10-25T18:47:05.671Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58411b4f3f851c0abc88000910c50c9d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.681Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-65d30bd9d48c68718085de5f: done: 0.023527 sec","job_status":"done","queue_duration_s":0.562763,"scheduling_latency_s":0.562787,"redis_calls":13,"redis_duration_s":0.003189,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000783,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001442,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006549156,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.023527,"completed_at":"2026-10-25T18:47:06.267Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002832,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1351gitlab # [ 162.158255] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1589da2e8bbafdaa6cc32fe1","created_at":"2026-10-25T18:47:05.687Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08edc248f5c521e0a496d2d02d07bd30","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.702Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1589da2e8bbafdaa6cc32fe1: start","job_status":"start","queue_duration_s":0.566069,"scheduling_latency_s":0.566082}1352gitlab # [ 162.208092] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.276Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1353gitlab # [ 162.208767] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.280Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1589da2e8bbafdaa6cc32fe1","created_at":"2026-10-25T18:47:05.687Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08edc248f5c521e0a496d2d02d07bd30","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.702Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1589da2e8bbafdaa6cc32fe1: done: 0.012256 sec","job_status":"done","queue_duration_s":0.566069,"scheduling_latency_s":0.566082,"redis_calls":10,"redis_duration_s":0.001979,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000382,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001211,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000386,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000269867,"duration_s":0.012256,"completed_at":"2026-10-25T18:47:06.280Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 162.253457] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"aa7d5ba2580e4b1f9c2a0797","created_at":"2026-10-25T18:47:05.710Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d5a43177eaddea7d1c1347713c42158","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.717Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-aa7d5ba2580e4b1f9c2a0797: start","job_status":"start","queue_duration_s":0.671041,"scheduling_latency_s":0.671085}1355gitlab # [ 162.302103] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.437Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"aa7d5ba2580e4b1f9c2a0797","created_at":"2026-10-25T18:47:05.710Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d5a43177eaddea7d1c1347713c42158","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.717Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-aa7d5ba2580e4b1f9c2a0797: done: 0.048733 sec","job_status":"done","queue_duration_s":0.671041,"scheduling_latency_s":0.671085,"redis_calls":12,"redis_duration_s":0.0029040000000000003,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001185,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001299,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000660698,"exclusive_lock_hold_duration_s":0.010209398,"duration_s":0.048733,"completed_at":"2026-10-25T18:47:06.437Z","load_balancing_strategy":"primary","db_duration_s":0.005306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 162.340609] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"866dfb65ff5b32a1753f0578","created_at":"2026-10-25T18:47:05.728Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cff2777f69b423e5f4a63c8e273cbd56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.735Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-866dfb65ff5b32a1753f0578: start","job_status":"start","queue_duration_s":0.740743,"scheduling_latency_s":0.740772}1357gitlab # [ 162.410322] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.545Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"866dfb65ff5b32a1753f0578","created_at":"2026-10-25T18:47:05.728Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cff2777f69b423e5f4a63c8e273cbd56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.735Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-866dfb65ff5b32a1753f0578: done: 0.069269 sec","job_status":"done","queue_duration_s":0.740743,"scheduling_latency_s":0.740772,"redis_calls":12,"redis_duration_s":0.004193,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000586,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001851,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001756,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000800102,"exclusive_lock_hold_duration_s":0.018198682,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.069269,"completed_at":"2026-10-25T18:47:06.545Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 162.468574] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.603Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dfae9dcc2bf317fe5d7686cd","created_at":"2026-10-25T18:47:05.739Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1866d101640b7a7c00cf7c68919b2e50","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.748Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dfae9dcc2bf317fe5d7686cd: start","job_status":"start","queue_duration_s":0.854718,"scheduling_latency_s":0.854745}1359gitlab # [ 162.538150] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"dfae9dcc2bf317fe5d7686cd","created_at":"2026-10-25T18:47:05.739Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1866d101640b7a7c00cf7c68919b2e50","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.748Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-dfae9dcc2bf317fe5d7686cd: done: 0.069734 sec","job_status":"done","queue_duration_s":0.854718,"scheduling_latency_s":0.854745,"redis_calls":10,"redis_duration_s":0.0033420000000000004,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00061,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043106,"duration_s":0.069734,"completed_at":"2026-10-25T18:47:06.672Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003713,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 162.595516] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.730Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"935a5e69e8edaeaf913ec81d","created_at":"2026-10-25T18:47:05.766Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"928276ea7b852129b3313c7edf62a755","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.769Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-935a5e69e8edaeaf913ec81d: start","job_status":"start","queue_duration_s":0.960842,"scheduling_latency_s":0.960884}1361gitlab # [ 162.628550] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"935a5e69e8edaeaf913ec81d","created_at":"2026-10-25T18:47:05.766Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"928276ea7b852129b3313c7edf62a755","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.769Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-935a5e69e8edaeaf913ec81d: done: 0.033022 sec","job_status":"done","queue_duration_s":0.960842,"scheduling_latency_s":0.960884,"redis_calls":12,"redis_duration_s":0.003577,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001309,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000633,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000479111,"duration_s":0.033022,"completed_at":"2026-10-25T18:47:06.763Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 162.669501] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"44bd4286d203ede9c31469d0","created_at":"2026-10-25T18:47:05.773Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2086f732e6e0e400cc5a164ede895fc2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-25T18:47:05.781Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-44bd4286d203ede9c31469d0: start","job_status":"start","queue_duration_s":1.02306,"scheduling_latency_s":1.023107}1363gitlab # [ 162.687097] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.822Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1364gitlab # [ 162.697064] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.832Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"44bd4286d203ede9c31469d0","created_at":"2026-10-25T18:47:05.773Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2086f732e6e0e400cc5a164ede895fc2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-25T18:47:05.781Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-44bd4286d203ede9c31469d0: done: 0.02765 sec","job_status":"done","queue_duration_s":1.02306,"scheduling_latency_s":1.023107,"redis_calls":10,"redis_duration_s":0.003015,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00062,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000550349,"duration_s":0.02765,"completed_at":"2026-10-25T18:47:06.832Z","load_balancing_strategy":"primary","db_duration_s":0.000784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 162.737133] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:06.872Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"6873fa8033bd2e7dff612485","created_at":"2026-10-25T18:47:05.791Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef660e82fb6b3a0e480d5e63be8eca8c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.800Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-6873fa8033bd2e7dff612485: start","job_status":"start","queue_duration_s":1.071626,"scheduling_latency_s":1.07167}1366gitlab # [ 163.039764] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.175Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"6873fa8033bd2e7dff612485","created_at":"2026-10-25T18:47:05.791Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef660e82fb6b3a0e480d5e63be8eca8c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.800Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-6873fa8033bd2e7dff612485: done: 0.302922 sec","job_status":"done","queue_duration_s":1.071626,"scheduling_latency_s":1.07167,"redis_calls":565,"redis_duration_s":0.099434,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.097686,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.215331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000289981,"duration_s":0.302922,"completed_at":"2026-10-25T18:47:07.175Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000858,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 163.127186] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.262Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fe11eb0e178079c7eca2331a","created_at":"2026-10-25T18:47:05.814Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a85dcda478e0936fbf07ffc860278ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.823Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fe11eb0e178079c7eca2331a: start","job_status":"start","queue_duration_s":1.438998,"scheduling_latency_s":1.439025}1368gitlab # [ 163.152507] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.288Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1369gitlab # [ 163.168449] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fe11eb0e178079c7eca2331a","created_at":"2026-10-25T18:47:05.814Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a85dcda478e0936fbf07ffc860278ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.823Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fe11eb0e178079c7eca2331a: done: 0.040322 sec","job_status":"done","queue_duration_s":1.438998,"scheduling_latency_s":1.439025,"redis_calls":10,"redis_duration_s":0.004052,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000367,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003228,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000458718,"duration_s":0.040322,"completed_at":"2026-10-25T18:47:07.302Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 163.208156] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f657aefbeb5eb4c0ab82e18e","created_at":"2026-10-25T18:47:05.831Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba133b8e519a8be5be7b0dd112b61a2b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.840Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f657aefbeb5eb4c0ab82e18e: start","job_status":"start","queue_duration_s":1.503185,"scheduling_latency_s":1.50323}1371gitlab # [ 163.254276] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f657aefbeb5eb4c0ab82e18e","created_at":"2026-10-25T18:47:05.831Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba133b8e519a8be5be7b0dd112b61a2b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.840Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f657aefbeb5eb4c0ab82e18e: done: 0.045181 sec","job_status":"done","queue_duration_s":1.503185,"scheduling_latency_s":1.50323,"redis_calls":10,"redis_duration_s":0.002761,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000638,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001608,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000522133,"duration_s":0.045181,"completed_at":"2026-10-25T18:47:07.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000981,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 163.294132] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9937d0fac5e965869b30759c","created_at":"2026-10-25T18:47:05.852Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"431af4a6644bd3b041a566dc10743431","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.861Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9937d0fac5e965869b30759c: start","job_status":"start","queue_duration_s":1.567625,"scheduling_latency_s":1.567653}1373gitlab # [ 163.313874] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.449Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1374gitlab # [ 163.327780] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.462Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9937d0fac5e965869b30759c","created_at":"2026-10-25T18:47:05.852Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"431af4a6644bd3b041a566dc10743431","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.861Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9937d0fac5e965869b30759c: done: 0.033268 sec","job_status":"done","queue_duration_s":1.567625,"scheduling_latency_s":1.567653,"redis_calls":10,"redis_duration_s":0.003237,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000688,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000636,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051515,"duration_s":0.033268,"completed_at":"2026-10-25T18:47:07.462Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000787,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 163.368846] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"39d60c97aed67593a271e89b","created_at":"2026-10-25T18:47:05.885Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"046d1002b1f46e951717b835c8b48ffd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.894Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-39d60c97aed67593a271e89b: start","job_status":"start","queue_duration_s":1.609803,"scheduling_latency_s":1.60984}1376gitlab # [ 163.432585] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.567Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"39d60c97aed67593a271e89b","created_at":"2026-10-25T18:47:05.885Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"046d1002b1f46e951717b835c8b48ffd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.894Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-39d60c97aed67593a271e89b: done: 0.063652 sec","job_status":"done","queue_duration_s":1.609803,"scheduling_latency_s":1.60984,"redis_calls":10,"redis_duration_s":0.002077,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001212,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000358146,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.063652,"completed_at":"2026-10-25T18:47:07.567Z","load_balancing_strategy":"primary","db_duration_s":0.018926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 163.481679] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.616Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b6e350513a9877adffdb2e29","created_at":"2026-10-25T18:47:05.906Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.915Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b6e350513a9877adffdb2e29: start","job_status":"start","queue_duration_s":1.700756,"scheduling_latency_s":1.700784}1378gitlab # [ 163.580661] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b6e350513a9877adffdb2e29","created_at":"2026-10-25T18:47:05.906Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.915Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b6e350513a9877adffdb2e29: done: 0.099708 sec","job_status":"done","queue_duration_s":1.700756,"scheduling_latency_s":1.700784,"redis_calls":71,"redis_duration_s":0.017558,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002983,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001042,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.012708,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000825,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000289423,"duration_s":0.099708,"completed_at":"2026-10-25T18:47:07.715Z","load_balancing_strategy":"primary","db_duration_s":0.002714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 163.657644] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.792Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1a3aec3825fa6c10ddf0ac2b","created_at":"2026-10-25T18:47:05.923Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb569caaa8d8c787ba2c7c5132a6d767","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.931Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1a3aec3825fa6c10ddf0ac2b: start","job_status":"start","queue_duration_s":1.861298,"scheduling_latency_s":1.861326}1380gitlab # [ 163.679425] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.815Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1381gitlab # [ 163.689891] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.824Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1a3aec3825fa6c10ddf0ac2b","created_at":"2026-10-25T18:47:05.923Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb569caaa8d8c787ba2c7c5132a6d767","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.931Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1a3aec3825fa6c10ddf0ac2b: done: 0.031853 sec","job_status":"done","queue_duration_s":1.861298,"scheduling_latency_s":1.861326,"redis_calls":10,"redis_duration_s":0.004383,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003869,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000231315,"duration_s":0.031853,"completed_at":"2026-10-25T18:47:07.824Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000919,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 163.729926] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b3fd73f41ce11910975c157c","created_at":"2026-10-25T18:47:05.942Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b305471370cd186dde359708ebc05d4b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-25T18:47:05.951Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b3fd73f41ce11910975c157c: start","job_status":"start","queue_duration_s":1.913483,"scheduling_latency_s":1.913514}1383gitlab # [ 163.747362] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.882Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1384gitlab # [ 163.758949] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b3fd73f41ce11910975c157c","created_at":"2026-10-25T18:47:05.942Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b305471370cd186dde359708ebc05d4b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-25T18:47:05.951Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b3fd73f41ce11910975c157c: done: 0.028977 sec","job_status":"done","queue_duration_s":1.913483,"scheduling_latency_s":1.913514,"redis_calls":10,"redis_duration_s":0.003517,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000659,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000596,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000483023,"duration_s":0.028977,"completed_at":"2026-10-25T18:47:07.894Z","load_balancing_strategy":"primary","db_duration_s":0.000725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 163.799073] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.934Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d2d4bd30dc4c5bf1b26348a1","created_at":"2026-10-25T18:47:05.963Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b839232c4a18f332785ef458ae6262f0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-25T18:47:05.973Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d2d4bd30dc4c5bf1b26348a1: start","job_status":"start","queue_duration_s":1.960894,"scheduling_latency_s":1.960946}1386gitlab # [ 163.849277] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:07.983Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d2d4bd30dc4c5bf1b26348a1","created_at":"2026-10-25T18:47:05.963Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b839232c4a18f332785ef458ae6262f0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-25T18:47:05.973Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d2d4bd30dc4c5bf1b26348a1: done: 0.049516 sec","job_status":"done","queue_duration_s":1.960894,"scheduling_latency_s":1.960946,"redis_calls":10,"redis_duration_s":0.003993999999999999,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002021,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000490845,"duration_s":0.049516,"completed_at":"2026-10-25T18:47:07.983Z","load_balancing_strategy":"primary","db_duration_s":0.005069,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 163.889578] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8c65dca4271367f45b49bb58","created_at":"2026-10-25T18:47:05.981Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.989Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8c65dca4271367f45b49bb58: start","job_status":"start","queue_duration_s":2.03505,"scheduling_latency_s":2.035094}1388gitlab # [ 163.944122] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.077Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8c65dca4271367f45b49bb58","created_at":"2026-10-25T18:47:05.981Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:05.989Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8c65dca4271367f45b49bb58: done: 0.053315 sec","job_status":"done","queue_duration_s":2.03505,"scheduling_latency_s":2.035094,"redis_calls":19,"redis_duration_s":0.006529,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001269,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000647,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003723,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000369879,"duration_s":0.053315,"completed_at":"2026-10-25T18:47:08.077Z","load_balancing_strategy":"primary","db_duration_s":0.002797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 163.982226] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9a225a44daf8c6bc9046aa49","created_at":"2026-10-25T18:47:06.011Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"156ce339479cc6e670e30b20542a97fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:06.022Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9a225a44daf8c6bc9046aa49: start","job_status":"start","queue_duration_s":2.057115,"scheduling_latency_s":2.057128}1390gitlab # [ 164.017909] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.153Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9a225a44daf8c6bc9046aa49","created_at":"2026-10-25T18:47:06.011Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-a699bab5d03c4094","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"156ce339479cc6e670e30b20542a97fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:06.022Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9a225a44daf8c6bc9046aa49: done: 0.073864 sec","job_status":"done","queue_duration_s":2.057115,"scheduling_latency_s":2.057128,"redis_calls":12,"redis_duration_s":0.003373,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00053,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001233,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000632203,"exclusive_lock_hold_duration_s":0.041426215,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.073864,"completed_at":"2026-10-25T18:47:08.153Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034164,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 164.039742] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05c21bce2553a5a3c44af28e","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05c21bce2553a5a3c44af28e: start","job_status":"start","queue_duration_s":0.446063,"scheduling_latency_s":0.446077}1392gitlab # [ 164.040506] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05c21bce2553a5a3c44af28e","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05c21bce2553a5a3c44af28e: done: 0.016051 sec","job_status":"done","queue_duration_s":0.446063,"scheduling_latency_s":0.446077,"redis_calls":16,"redis_duration_s":0.003246,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000492,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001866,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000249193,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016051,"completed_at":"2026-10-25T18:47:08.170Z","load_balancing_strategy":"primary","db_duration_s":0.000526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 164.066193] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba361b6a92370a05ce9f7acd","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba361b6a92370a05ce9f7acd: start","job_status":"start","queue_duration_s":0.463138,"scheduling_latency_s":0.46315}1394gitlab # [ 164.147428] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba361b6a92370a05ce9f7acd","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba361b6a92370a05ce9f7acd: done: 0.11074 sec","job_status":"done","queue_duration_s":0.463138,"scheduling_latency_s":0.46315,"redis_calls":13,"redis_duration_s":0.092456,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.011876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.08058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.11074,"completed_at":"2026-10-25T18:47:08.282Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 164.184530] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5613220d83800a5fe596058f","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5613220d83800a5fe596058f: start","job_status":"start","queue_duration_s":0.610951,"scheduling_latency_s":0.610979}1396gitlab # [ 164.206812] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5613220d83800a5fe596058f","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5613220d83800a5fe596058f: done: 0.022295 sec","job_status":"done","queue_duration_s":0.610951,"scheduling_latency_s":0.610979,"redis_calls":13,"redis_duration_s":0.005072,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022295,"completed_at":"2026-10-25T18:47:08.341Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 164.244093] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b1c5e1ae4233cce3e532b2d","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b1c5e1ae4233cce3e532b2d: start","job_status":"start","queue_duration_s":0.670768,"scheduling_latency_s":0.670806}1398gitlab # [ 164.266340] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b1c5e1ae4233cce3e532b2d","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b1c5e1ae4233cce3e532b2d: done: 0.022147 sec","job_status":"done","queue_duration_s":0.670768,"scheduling_latency_s":0.670806,"redis_calls":13,"redis_duration_s":0.0037949999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001171,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002624,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022147,"completed_at":"2026-10-25T18:47:08.401Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 164.304314] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba26a3bf9ac25b98b16f573f","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba26a3bf9ac25b98b16f573f: start","job_status":"start","queue_duration_s":0.730852,"scheduling_latency_s":0.730899}1400gitlab # [ 164.330229] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba26a3bf9ac25b98b16f573f","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba26a3bf9ac25b98b16f573f: done: 0.025964 sec","job_status":"done","queue_duration_s":0.730852,"scheduling_latency_s":0.730899,"redis_calls":13,"redis_duration_s":0.0038650000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001273,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002592,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025964,"completed_at":"2026-10-25T18:47:08.465Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 164.367427] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5b5f431a1157b570f509fac","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5b5f431a1157b570f509fac: start","job_status":"start","queue_duration_s":0.79397,"scheduling_latency_s":0.794008}1402gitlab # [ 164.400263] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5b5f431a1157b570f509fac","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5b5f431a1157b570f509fac: done: 0.032718 sec","job_status":"done","queue_duration_s":0.79397,"scheduling_latency_s":0.794008,"redis_calls":13,"redis_duration_s":0.002729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032718,"completed_at":"2026-10-25T18:47:08.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 164.437152] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcfbe82378d6e546f4498abb","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcfbe82378d6e546f4498abb: start","job_status":"start","queue_duration_s":0.863853,"scheduling_latency_s":0.863892}1404gitlab # [ 164.462141] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcfbe82378d6e546f4498abb","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcfbe82378d6e546f4498abb: done: 0.024871 sec","job_status":"done","queue_duration_s":0.863853,"scheduling_latency_s":0.863892,"redis_calls":13,"redis_duration_s":0.003908,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002651,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024871,"completed_at":"2026-10-25T18:47:08.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 164.499697] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ccf9c1ed97281940568398b","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ccf9c1ed97281940568398b: start","job_status":"start","queue_duration_s":0.925649,"scheduling_latency_s":0.925696}1406gitlab # [ 164.523769] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ccf9c1ed97281940568398b","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ccf9c1ed97281940568398b: done: 0.024651 sec","job_status":"done","queue_duration_s":0.925649,"scheduling_latency_s":0.925696,"redis_calls":13,"redis_duration_s":0.003698,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001218,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00248,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024651,"completed_at":"2026-10-25T18:47:08.658Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 164.560460] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3ecb7f0cd1bded1d69e19da","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3ecb7f0cd1bded1d69e19da: start","job_status":"start","queue_duration_s":0.987,"scheduling_latency_s":0.987038}1408gitlab # [ 164.593201] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3ecb7f0cd1bded1d69e19da","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3ecb7f0cd1bded1d69e19da: done: 0.032723 sec","job_status":"done","queue_duration_s":0.987,"scheduling_latency_s":0.987038,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032723,"completed_at":"2026-10-25T18:47:08.728Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 164.630696] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14ca5a7356d780c537aeb2f1","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14ca5a7356d780c537aeb2f1: start","job_status":"start","queue_duration_s":1.057238,"scheduling_latency_s":1.057338}1410gitlab # [ 164.653822] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14ca5a7356d780c537aeb2f1","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14ca5a7356d780c537aeb2f1: done: 0.023014 sec","job_status":"done","queue_duration_s":1.057238,"scheduling_latency_s":1.057338,"redis_calls":13,"redis_duration_s":0.003845,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001156,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002689,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023014,"completed_at":"2026-10-25T18:47:08.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 164.691147] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddc8e6d53e6201474764295c","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddc8e6d53e6201474764295c: start","job_status":"start","queue_duration_s":1.117737,"scheduling_latency_s":1.117784}1412gitlab # [ 164.717117] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddc8e6d53e6201474764295c","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddc8e6d53e6201474764295c: done: 0.025981 sec","job_status":"done","queue_duration_s":1.117737,"scheduling_latency_s":1.117784,"redis_calls":13,"redis_duration_s":0.003843,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002584,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025981,"completed_at":"2026-10-25T18:47:08.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 164.753459] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a59ccc4bdb90417de288da7e","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a59ccc4bdb90417de288da7e: start","job_status":"start","queue_duration_s":1.179983,"scheduling_latency_s":1.180021}1414gitlab # [ 164.785971] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a59ccc4bdb90417de288da7e","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a59ccc4bdb90417de288da7e: done: 0.032475 sec","job_status":"done","queue_duration_s":1.179983,"scheduling_latency_s":1.180021,"redis_calls":13,"redis_duration_s":0.002627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001746,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032475,"completed_at":"2026-10-25T18:47:08.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 164.822500] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e89be41638ad91141934e98","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e89be41638ad91141934e98: start","job_status":"start","queue_duration_s":1.249043,"scheduling_latency_s":1.24908}1416gitlab # [ 164.854837] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:08.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e89be41638ad91141934e98","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e89be41638ad91141934e98: done: 0.032235 sec","job_status":"done","queue_duration_s":1.249043,"scheduling_latency_s":1.24908,"redis_calls":13,"redis_duration_s":0.002511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001654,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032235,"completed_at":"2026-10-25T18:47:08.989Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 164.891492] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c672d5eb56faab63dd37d496","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c672d5eb56faab63dd37d496: start","job_status":"start","queue_duration_s":1.318022,"scheduling_latency_s":1.318067}1418gitlab # [ 164.914953] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c672d5eb56faab63dd37d496","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c672d5eb56faab63dd37d496: done: 0.023362 sec","job_status":"done","queue_duration_s":1.318022,"scheduling_latency_s":1.318067,"redis_calls":13,"redis_duration_s":0.0037489999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001178,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002571,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023362,"completed_at":"2026-10-25T18:47:09.049Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 164.952370] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"009a124565bbf4041621c885","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-009a124565bbf4041621c885: start","job_status":"start","queue_duration_s":1.378903,"scheduling_latency_s":1.378952}1420gitlab # [ 164.979100] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"009a124565bbf4041621c885","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-009a124565bbf4041621c885: done: 0.02671 sec","job_status":"done","queue_duration_s":1.378903,"scheduling_latency_s":1.378952,"redis_calls":13,"redis_duration_s":0.004006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001291,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002715,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02671,"completed_at":"2026-10-25T18:47:09.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 165.016160] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5971f62d1948f1c3fde105cb","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5971f62d1948f1c3fde105cb: start","job_status":"start","queue_duration_s":1.442358,"scheduling_latency_s":1.442397}1422gitlab # [ 165.049450] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5971f62d1948f1c3fde105cb","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5971f62d1948f1c3fde105cb: done: 0.033693 sec","job_status":"done","queue_duration_s":1.442358,"scheduling_latency_s":1.442397,"redis_calls":13,"redis_duration_s":0.002773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001787,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033693,"completed_at":"2026-10-25T18:47:09.184Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 165.070843] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f47552b4dd9bd3aecc24eac8","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f47552b4dd9bd3aecc24eac8: start","job_status":"start","queue_duration_s":1.478594,"scheduling_latency_s":1.478625}1424gitlab # [ 165.072533] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f47552b4dd9bd3aecc24eac8","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f47552b4dd9bd3aecc24eac8: done: 0.020603 sec","job_status":"done","queue_duration_s":1.478594,"scheduling_latency_s":1.478625,"redis_calls":13,"redis_duration_s":0.0036620000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001185,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002477,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020603,"completed_at":"2026-10-25T18:47:09.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 165.133390] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd61f6a2a53a1c60a9973865","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd61f6a2a53a1c60a9973865: start","job_status":"start","queue_duration_s":1.55989,"scheduling_latency_s":1.559936}1426gitlab # [ 165.166791] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd61f6a2a53a1c60a9973865","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd61f6a2a53a1c60a9973865: done: 0.033279 sec","job_status":"done","queue_duration_s":1.55989,"scheduling_latency_s":1.559936,"redis_calls":13,"redis_duration_s":0.0028610000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033279,"completed_at":"2026-10-25T18:47:09.301Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 165.203332] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"951b8f7b6179eb525a49fa62","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-951b8f7b6179eb525a49fa62: start","job_status":"start","queue_duration_s":1.629903,"scheduling_latency_s":1.629941}1428gitlab # [ 165.234840] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"951b8f7b6179eb525a49fa62","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-951b8f7b6179eb525a49fa62: done: 0.031453 sec","job_status":"done","queue_duration_s":1.629903,"scheduling_latency_s":1.629941,"redis_calls":13,"redis_duration_s":0.002443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001667,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031453,"completed_at":"2026-10-25T18:47:09.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 165.272498] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b555836a1f27715bb2ca8f8d","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b555836a1f27715bb2ca8f8d: start","job_status":"start","queue_duration_s":1.699024,"scheduling_latency_s":1.699069}1430gitlab # [ 165.298778] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b555836a1f27715bb2ca8f8d","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b555836a1f27715bb2ca8f8d: done: 0.026201 sec","job_status":"done","queue_duration_s":1.699024,"scheduling_latency_s":1.699069,"redis_calls":13,"redis_duration_s":0.003936,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001246,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00269,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026201,"completed_at":"2026-10-25T18:47:09.433Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 165.335408] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bd31ff33ba67ba86eaadb14","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bd31ff33ba67ba86eaadb14: start","job_status":"start","queue_duration_s":1.761959,"scheduling_latency_s":1.761998}1432gitlab # [ 165.360745] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bd31ff33ba67ba86eaadb14","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bd31ff33ba67ba86eaadb14: done: 0.025216 sec","job_status":"done","queue_duration_s":1.761959,"scheduling_latency_s":1.761998,"redis_calls":13,"redis_duration_s":0.004011,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002724,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025216,"completed_at":"2026-10-25T18:47:09.495Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 165.397530] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb54ddbc04cce7c907483196","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb54ddbc04cce7c907483196: start","job_status":"start","queue_duration_s":1.824196,"scheduling_latency_s":1.824233}1434gitlab # [ 165.430830] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb54ddbc04cce7c907483196","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb54ddbc04cce7c907483196: done: 0.032819 sec","job_status":"done","queue_duration_s":1.824196,"scheduling_latency_s":1.824233,"redis_calls":13,"redis_duration_s":0.002768,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032819,"completed_at":"2026-10-25T18:47:09.565Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 165.467065] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00bd8b0dadd14272ad26b4bd","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00bd8b0dadd14272ad26b4bd: start","job_status":"start","queue_duration_s":1.893695,"scheduling_latency_s":1.893739}1436gitlab # [ 165.504575] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00bd8b0dadd14272ad26b4bd","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00bd8b0dadd14272ad26b4bd: done: 0.036301 sec","job_status":"done","queue_duration_s":1.893695,"scheduling_latency_s":1.893739,"redis_calls":13,"redis_duration_s":0.004502,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00297,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036301,"completed_at":"2026-10-25T18:47:09.638Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 165.540427] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"024af451a965d6f5db521801","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-024af451a965d6f5db521801: start","job_status":"start","queue_duration_s":1.966985,"scheduling_latency_s":1.967024}1438gitlab # [ 165.572549] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"024af451a965d6f5db521801","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-024af451a965d6f5db521801: done: 0.032044 sec","job_status":"done","queue_duration_s":1.966985,"scheduling_latency_s":1.967024,"redis_calls":13,"redis_duration_s":0.0026160000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001806,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032044,"completed_at":"2026-10-25T18:47:09.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 165.610782] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3f9416aa31a8eabd5a38293","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3f9416aa31a8eabd5a38293: start","job_status":"start","queue_duration_s":2.037337,"scheduling_latency_s":2.037383}1440gitlab # [ 165.638093] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3f9416aa31a8eabd5a38293","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3f9416aa31a8eabd5a38293: done: 0.027279 sec","job_status":"done","queue_duration_s":2.037337,"scheduling_latency_s":2.037383,"redis_calls":13,"redis_duration_s":0.00404,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002691,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027279,"completed_at":"2026-10-25T18:47:09.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 165.675451] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a31daaf2b0453117e31de6ce","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a31daaf2b0453117e31de6ce: start","job_status":"start","queue_duration_s":2.101534,"scheduling_latency_s":2.101574}1442gitlab # [ 165.710443] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a31daaf2b0453117e31de6ce","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a31daaf2b0453117e31de6ce: done: 0.03515 sec","job_status":"done","queue_duration_s":2.101534,"scheduling_latency_s":2.101574,"redis_calls":13,"redis_duration_s":0.002786,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001841,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03515,"completed_at":"2026-10-25T18:47:09.845Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 165.746722] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"655b10c11392dfe1706f8ff0","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-655b10c11392dfe1706f8ff0: start","job_status":"start","queue_duration_s":2.173206,"scheduling_latency_s":2.173244}1444gitlab # [ 165.781679] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"655b10c11392dfe1706f8ff0","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-655b10c11392dfe1706f8ff0: done: 0.034806 sec","job_status":"done","queue_duration_s":2.173206,"scheduling_latency_s":2.173244,"redis_calls":13,"redis_duration_s":0.002717,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001841,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034806,"completed_at":"2026-10-25T18:47:09.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 165.819097] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c14860e094bf1778464be85","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c14860e094bf1778464be85: start","job_status":"start","queue_duration_s":2.245612,"scheduling_latency_s":2.245658}1446gitlab # [ 165.844374] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:09.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c14860e094bf1778464be85","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c14860e094bf1778464be85: done: 0.025281 sec","job_status":"done","queue_duration_s":2.245612,"scheduling_latency_s":2.245658,"redis_calls":13,"redis_duration_s":0.0038780000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00124,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002638,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025281,"completed_at":"2026-10-25T18:47:09.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 165.881124] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c12284715bc47165c45d313e","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c12284715bc47165c45d313e: start","job_status":"start","queue_duration_s":2.307868,"scheduling_latency_s":2.307896}1448gitlab # [ 165.905949] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c12284715bc47165c45d313e","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c12284715bc47165c45d313e: done: 0.0246 sec","job_status":"done","queue_duration_s":2.307868,"scheduling_latency_s":2.307896,"redis_calls":13,"redis_duration_s":0.0038120000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001233,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002579,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0246,"completed_at":"2026-10-25T18:47:10.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 165.942847] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e75b9f188c2346c4783f0de1","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e75b9f188c2346c4783f0de1: start","job_status":"start","queue_duration_s":2.369724,"scheduling_latency_s":2.369747}1450gitlab # [ 165.979496] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e75b9f188c2346c4783f0de1","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e75b9f188c2346c4783f0de1: done: 0.036084 sec","job_status":"done","queue_duration_s":2.369724,"scheduling_latency_s":2.369747,"redis_calls":13,"redis_duration_s":0.004139,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001227,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036084,"completed_at":"2026-10-25T18:47:10.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 166.023071] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee7f58ac9c40c2e36345a78","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee7f58ac9c40c2e36345a78: start","job_status":"start","queue_duration_s":2.449745,"scheduling_latency_s":2.449785}1452gitlab # [ 166.150629] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee7f58ac9c40c2e36345a78","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee7f58ac9c40c2e36345a78: done: 0.127269 sec","job_status":"done","queue_duration_s":2.449745,"scheduling_latency_s":2.449785,"redis_calls":13,"redis_duration_s":0.105967,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.104091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.127269,"completed_at":"2026-10-25T18:47:10.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 166.208261] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5de612f8ec3a4a2ffb9dac0","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5de612f8ec3a4a2ffb9dac0: start","job_status":"start","queue_duration_s":2.635121,"scheduling_latency_s":2.635148}1454gitlab # [ 166.244128] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5de612f8ec3a4a2ffb9dac0","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5de612f8ec3a4a2ffb9dac0: done: 0.035311 sec","job_status":"done","queue_duration_s":2.635121,"scheduling_latency_s":2.635148,"redis_calls":13,"redis_duration_s":0.003934,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001275,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002659,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035311,"completed_at":"2026-10-25T18:47:10.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 166.281072] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4687b9aefa32d27cc8aaafbe","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4687b9aefa32d27cc8aaafbe: start","job_status":"start","queue_duration_s":2.707869,"scheduling_latency_s":2.707893}1456gitlab # [ 166.317436] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4687b9aefa32d27cc8aaafbe","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4687b9aefa32d27cc8aaafbe: done: 0.035991 sec","job_status":"done","queue_duration_s":2.707869,"scheduling_latency_s":2.707893,"redis_calls":13,"redis_duration_s":0.0038590000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001309,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00255,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035991,"completed_at":"2026-10-25T18:47:10.452Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 166.356348] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20ae9fe69f3edf65fff8901e","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20ae9fe69f3edf65fff8901e: start","job_status":"start","queue_duration_s":2.782943,"scheduling_latency_s":2.782982}1458gitlab # [ 166.388994] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20ae9fe69f3edf65fff8901e","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20ae9fe69f3edf65fff8901e: done: 0.032225 sec","job_status":"done","queue_duration_s":2.782943,"scheduling_latency_s":2.782982,"redis_calls":13,"redis_duration_s":0.002498,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001682,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032225,"completed_at":"2026-10-25T18:47:10.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 166.425889] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81c90be3b178f320238d765a","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81c90be3b178f320238d765a: start","job_status":"start","queue_duration_s":2.852506,"scheduling_latency_s":2.852558}1460gitlab # [ 166.458295] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81c90be3b178f320238d765a","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81c90be3b178f320238d765a: done: 0.031972 sec","job_status":"done","queue_duration_s":2.852506,"scheduling_latency_s":2.852558,"redis_calls":13,"redis_duration_s":0.0025859999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001642,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031972,"completed_at":"2026-10-25T18:47:10.592Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 166.495489] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0242c3605f1398dcf7ac28b8","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0242c3605f1398dcf7ac28b8: start","job_status":"start","queue_duration_s":2.921931,"scheduling_latency_s":2.921967}1462gitlab # [ 166.528536] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0242c3605f1398dcf7ac28b8","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0242c3605f1398dcf7ac28b8: done: 0.032288 sec","job_status":"done","queue_duration_s":2.921931,"scheduling_latency_s":2.921967,"redis_calls":13,"redis_duration_s":0.00245,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001654,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032288,"completed_at":"2026-10-25T18:47:10.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 166.565518] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be147f5213f8ab9f746ef670","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be147f5213f8ab9f746ef670: start","job_status":"start","queue_duration_s":2.992147,"scheduling_latency_s":2.992184}1464gitlab # [ 166.589701] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be147f5213f8ab9f746ef670","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be147f5213f8ab9f746ef670: done: 0.024041 sec","job_status":"done","queue_duration_s":2.992147,"scheduling_latency_s":2.992184,"redis_calls":13,"redis_duration_s":0.0038970000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001243,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002654,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024041,"completed_at":"2026-10-25T18:47:10.724Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 166.627134] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb2d81205a7c5cf42736f4d1","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb2d81205a7c5cf42736f4d1: start","job_status":"start","queue_duration_s":3.053816,"scheduling_latency_s":3.053854}1466gitlab # [ 166.659279] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb2d81205a7c5cf42736f4d1","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb2d81205a7c5cf42736f4d1: done: 0.031947 sec","job_status":"done","queue_duration_s":3.053816,"scheduling_latency_s":3.053854,"redis_calls":13,"redis_duration_s":0.002439,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001621,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031947,"completed_at":"2026-10-25T18:47:10.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 166.696550] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c6743c993ab4f56b4323e1c","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c6743c993ab4f56b4323e1c: start","job_status":"start","queue_duration_s":3.122841,"scheduling_latency_s":3.122879}1468gitlab # [ 166.728371] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c6743c993ab4f56b4323e1c","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c6743c993ab4f56b4323e1c: done: 0.031961 sec","job_status":"done","queue_duration_s":3.122841,"scheduling_latency_s":3.122879,"redis_calls":13,"redis_duration_s":0.002587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001709,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031961,"completed_at":"2026-10-25T18:47:10.863Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 166.765402] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8caaf81f242d402ecc0252d3","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8caaf81f242d402ecc0252d3: start","job_status":"start","queue_duration_s":3.191881,"scheduling_latency_s":3.191918}1470gitlab # [ 166.797483] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8caaf81f242d402ecc0252d3","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8caaf81f242d402ecc0252d3: done: 0.032093 sec","job_status":"done","queue_duration_s":3.191881,"scheduling_latency_s":3.191918,"redis_calls":13,"redis_duration_s":0.00253,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000811,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032093,"completed_at":"2026-10-25T18:47:10.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 166.834128] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38f1e548bb2275d692c70690","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38f1e548bb2275d692c70690: start","job_status":"start","queue_duration_s":3.260784,"scheduling_latency_s":3.260821}1472gitlab # [ 166.858731] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:10.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38f1e548bb2275d692c70690","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38f1e548bb2275d692c70690: done: 0.024426 sec","job_status":"done","queue_duration_s":3.260784,"scheduling_latency_s":3.260821,"redis_calls":13,"redis_duration_s":0.0038250000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001231,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002594,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024426,"completed_at":"2026-10-25T18:47:10.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 166.896363] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0afe26c6e1ba2807d9e977b6","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0afe26c6e1ba2807d9e977b6: start","job_status":"start","queue_duration_s":3.322931,"scheduling_latency_s":3.322968}1474gitlab # [ 166.928285] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0afe26c6e1ba2807d9e977b6","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0afe26c6e1ba2807d9e977b6: done: 0.031815 sec","job_status":"done","queue_duration_s":3.322931,"scheduling_latency_s":3.322968,"redis_calls":13,"redis_duration_s":0.002449,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001633,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031815,"completed_at":"2026-10-25T18:47:11.063Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 166.965151] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5eef1f950570b0bd0f44efde","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5eef1f950570b0bd0f44efde: start","job_status":"start","queue_duration_s":3.391785,"scheduling_latency_s":3.391823}1476gitlab # [ 166.997489] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5eef1f950570b0bd0f44efde","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5eef1f950570b0bd0f44efde: done: 0.031896 sec","job_status":"done","queue_duration_s":3.391785,"scheduling_latency_s":3.391823,"redis_calls":13,"redis_duration_s":0.002484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000839,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001645,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031896,"completed_at":"2026-10-25T18:47:11.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 167.035049] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe0e4fea8f621827a37512a2","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe0e4fea8f621827a37512a2: start","job_status":"start","queue_duration_s":3.461546,"scheduling_latency_s":3.461596}1478gitlab # [ 167.063148] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe0e4fea8f621827a37512a2","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe0e4fea8f621827a37512a2: done: 0.028054 sec","job_status":"done","queue_duration_s":3.461546,"scheduling_latency_s":3.461596,"redis_calls":13,"redis_duration_s":0.004111,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001267,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002844,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028054,"completed_at":"2026-10-25T18:47:11.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 167.082597] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2783f50f4e32e85f767b194f","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2783f50f4e32e85f767b194f: start","job_status":"start","queue_duration_s":3.509129,"scheduling_latency_s":3.509171}1480gitlab # [ 167.107987] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2783f50f4e32e85f767b194f","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2783f50f4e32e85f767b194f: done: 0.025073 sec","job_status":"done","queue_duration_s":3.509129,"scheduling_latency_s":3.509171,"redis_calls":13,"redis_duration_s":0.003874,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002682,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025073,"completed_at":"2026-10-25T18:47:11.242Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 167.111197] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7916105f08e118e69ce4db9","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7916105f08e118e69ce4db9: start","job_status":"start","queue_duration_s":3.538044,"scheduling_latency_s":3.53807}1482gitlab # [ 167.132442] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7916105f08e118e69ce4db9","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7916105f08e118e69ce4db9: done: 0.020924 sec","job_status":"done","queue_duration_s":3.538044,"scheduling_latency_s":3.53807,"redis_calls":13,"redis_duration_s":0.003248,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020924,"completed_at":"2026-10-25T18:47:11.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 167.179192] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d54341de7218ab11d864afc","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d54341de7218ab11d864afc: start","job_status":"start","queue_duration_s":3.565643,"scheduling_latency_s":3.565657}1484gitlab # [ 167.180150] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d54341de7218ab11d864afc","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d54341de7218ab11d864afc: done: 0.011538 sec","job_status":"done","queue_duration_s":3.565643,"scheduling_latency_s":3.565657,"redis_calls":13,"redis_duration_s":0.0027300000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011538,"completed_at":"2026-10-25T18:47:11.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 167.198918] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6b3e194c9208c67f8d153d5","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6b3e194c9208c67f8d153d5: start","job_status":"start","queue_duration_s":3.578474,"scheduling_latency_s":3.578488}1486gitlab # [ 167.245028] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6b3e194c9208c67f8d153d5","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6b3e194c9208c67f8d153d5: done: 0.010407 sec","job_status":"done","queue_duration_s":3.578474,"scheduling_latency_s":3.578488,"redis_calls":13,"redis_duration_s":0.002484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001666,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010407,"completed_at":"2026-10-25T18:47:11.297Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 167.246041] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c8b40a379e9c62038c96b74","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c8b40a379e9c62038c96b74: start","job_status":"start","queue_duration_s":3.590098,"scheduling_latency_s":3.590111}1488gitlab # [ 167.255035] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c8b40a379e9c62038c96b74","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c8b40a379e9c62038c96b74: done: 0.010376 sec","job_status":"done","queue_duration_s":3.590098,"scheduling_latency_s":3.590111,"redis_calls":13,"redis_duration_s":0.0024549999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001707,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010376,"completed_at":"2026-10-25T18:47:11.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 167.306133] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"911fadd1242e1f08235ccb21","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-911fadd1242e1f08235ccb21: start","job_status":"start","queue_duration_s":3.601537,"scheduling_latency_s":3.601551}1490gitlab # [ 167.306962] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"911fadd1242e1f08235ccb21","created_at":"2026-10-25T18:47:07.647Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8fdc1784a5ca41df","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"db64ec7b6640ade42ac5ecf99b0652a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:07.708Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-911fadd1242e1f08235ccb21: done: 0.020406 sec","job_status":"done","queue_duration_s":3.601537,"scheduling_latency_s":3.601551,"redis_calls":13,"redis_duration_s":0.012060999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005417,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006644,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020406,"completed_at":"2026-10-25T18:47:11.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 167.345584] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.333Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1e1044161f7d4813c9e82ef7","created_at":"2026-10-25T18:47:08.050Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8dcacbd9cebf484f","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:08.067Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1e1044161f7d4813c9e82ef7: start","job_status":"start","queue_duration_s":3.266014,"scheduling_latency_s":3.266028}1492gitlab # [ 167.346502] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.348Z","message":"No existing merge request to be cleaned up."}1493gitlab # [ 167.347318] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.356Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1e1044161f7d4813c9e82ef7","created_at":"2026-10-25T18:47:08.050Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8dcacbd9cebf484f","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:08.067Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1e1044161f7d4813c9e82ef7: done: 0.023017 sec","job_status":"done","queue_duration_s":3.266014,"scheduling_latency_s":3.266028,"redis_calls":13,"redis_duration_s":0.003669,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000764,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00152,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001385,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208686,"duration_s":0.023017,"completed_at":"2026-10-25T18:47:11.356Z","load_balancing_strategy":"primary","db_duration_s":0.004653,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494gitlab # [ 167.417478] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.357Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"140a8b65e4183fa7166974a9","created_at":"2026-10-25T18:47:08.050Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8dcacbd9cebf484f","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:08.067Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-140a8b65e4183fa7166974a9: start","job_status":"start","queue_duration_s":3.290373,"scheduling_latency_s":3.290385}1495gitlab # [ 167.418271] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.364Z","message":"No existing merge request to be cleaned up."}1496gitlab # [ 167.419174] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.369Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"140a8b65e4183fa7166974a9","created_at":"2026-10-25T18:47:08.050Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8dcacbd9cebf484f","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:08.067Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-140a8b65e4183fa7166974a9: done: 0.011899 sec","job_status":"done","queue_duration_s":3.290373,"scheduling_latency_s":3.290385,"redis_calls":10,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011899,"completed_at":"2026-10-25T18:47:11.369Z","load_balancing_strategy":"primary","db_duration_s":0.000987,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 167.464660] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.370Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"73125e2c1da2abbfcf803ae8","created_at":"2026-10-25T18:47:08.050Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8dcacbd9cebf484f","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:08.067Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-73125e2c1da2abbfcf803ae8: start","job_status":"start","queue_duration_s":3.303108,"scheduling_latency_s":3.303121}1498gitlab # [ 167.465484] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.376Z","message":"No existing merge request to be cleaned up."}1499gitlab # [ 167.466283] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.380Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"73125e2c1da2abbfcf803ae8","created_at":"2026-10-25T18:47:08.050Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8dcacbd9cebf484f","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:08.067Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-73125e2c1da2abbfcf803ae8: done: 0.010525 sec","job_status":"done","queue_duration_s":3.303108,"scheduling_latency_s":3.303121,"redis_calls":10,"redis_duration_s":0.001242,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000758,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010525,"completed_at":"2026-10-25T18:47:11.380Z","load_balancing_strategy":"primary","db_duration_s":0.000917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 167.478828] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.386Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"40d24a661ae164b36fffe5d4","created_at":"2026-10-25T18:47:08.050Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8dcacbd9cebf484f","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:08.067Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-40d24a661ae164b36fffe5d4: start","job_status":"start","queue_duration_s":3.31896,"scheduling_latency_s":3.318975}1501gitlab # [ 167.527396] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.400Z","message":"No existing merge request to be cleaned up."}1502gitlab # [ 167.528916] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:11.406Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"40d24a661ae164b36fffe5d4","created_at":"2026-10-25T18:47:08.050Z","trace_propagation_headers":{"sentry-trace":"06ec4b804e62415ca61cd663d42fd4c1-8dcacbd9cebf484f","baggage":"sentry-trace_id=06ec4b804e62415ca61cd663d42fd4c1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"dc0b917302738a7eb7defa60d38dc723","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:08.067Z","job_size_bytes":2,"pid":1616,"sidekiq_tid":"madg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-40d24a661ae164b36fffe5d4: done: 0.020338 sec","job_status":"done","queue_duration_s":3.31896,"scheduling_latency_s":3.318975,"redis_calls":10,"redis_duration_s":0.005063,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003415,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020338,"completed_at":"2026-10-25T18:47:11.406Z","load_balancing_strategy":"primary","db_duration_s":0.008665,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 179.956708] postgres[2401]: [2401] ERROR: relation "table_without_models" does not exist1504gitlab # [ 179.959550] postgres[2401]: [2401] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1505gitlab # [ 183.436112] runuser[2203]: pam_unix(runuser:session): session closed for user gitlab1506gitlab # [ 183.437516] setup-gitlab-start[2178]: + systemctl restart gitlab-sidekiq.service1507gitlab # [ 183.459890] systemd[1]: Stopping gitlab-sidekiq.service...1508gitlab # [ 183.462382] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:27.597Z","message":"Shutting down"}1509gitlab # [ 183.463432] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:27.597Z","message":"Scheduler exiting..."}1510gitlab # [ 183.464188] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:27.597Z","message":"Terminating quiet threads for default capsule"}1511gitlab # [ 183.464985] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:27.598Z","message":"Scheduler exiting..."}1512gitlab # [ 183.969155] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:28.104Z","message":"Pausing to allow jobs to finish..."}1513gitlab # [ 187.473951] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:31.608Z","message":"Bye!"}1514gitlab # [ 187.497148] sidekiq-cluster[1616]: {"severity":"INFO","time":"2026-10-25T18:47:31.632Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1616,"worker_id":"sidekiq_0","memwd_rss_bytes":927989760,"retry":0}1515gitlab # [ 187.884963] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1516gitlab # [ 187.886660] systemd[1]: Stopped gitlab-sidekiq.service.1517gitlab # [ 187.888145] systemd[1]: gitlab-sidekiq.service: Consumed 55.642s CPU time over 1min 55.057s wall clock time, 957.8M memory peak, 4.4M read from disk.1518gitlab # [ 187.896622] systemd[1]: Started gitlab-sidekiq.service.1519gitlab # [ 187.899299] setup-gitlab-start[2178]: + mkdir -p /var/lib/secrets1520gitlab # [ 187.902360] setup-gitlab-start[2178]: + echo nixbot-test-token-of-20-chars1521gitlab # [ 187.904186] setup-gitlab-start[2178]: + chmod 644 /var/lib/secrets/gitlab-token1522gitlab # [ 187.906205] setup-gitlab-start[2178]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1523gitlab # [ 188.415511] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:32 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 4801524gitlab # [ 188.925272] gitaly[904]: time="2026-10-25T18:47:33.060Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.059" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1525gitlab # [ 188.927224] gitaly[904]: time="2026-10-25T18:47:33.062Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.059" grpc.time_ms=2.475 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1526gitlab # [ 189.075131] gitaly[904]: time="2026-10-25T18:47:33.210Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.159" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.210" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1527gitlab # [ 189.097845] gitaly[904]: time="2026-10-25T18:47:33.211Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.159" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.210" grpc.time_ms=0.343 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1528gitlab # [ 189.153955] gitaly[904]: time="2026-10-25T18:47:33.289Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 diskcache=5403fda2-0cad-4bb9-8c2c-689b574a9ff6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-25T18:47:43.191" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.242" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1529gitlab # [ 189.155139] gitaly[904]: time="2026-10-25T18:47:33.290Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1577 command.oublock=32 command.real_time_ms=19 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-25T18:47:43.191" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.242" grpc.time_ms=47.654 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1530gitlab # [ 189.200478] gitaly[904]: time="2026-10-25T18:47:33.335Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.285" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.335" pid=904 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1531gitlab # [ 189.201728] gitaly[904]: time="2026-10-25T18:47:33.336Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.285" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.335" grpc.time_ms=0.429 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1532gitlab # [ 189.221139] gitaly[904]: time="2026-10-25T18:47:33.356Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-25T18:47:43.294" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.345" grpc.time_ms=9.704 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1533gitlab # [ 189.284415] gitaly[904]: time="2026-10-25T18:47:33.420Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.419" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1534gitlab # [ 189.285651] gitaly[904]: time="2026-10-25T18:47:33.420Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.419" grpc.time_ms=0.456 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1535gitlab # [ 189.370713] gitaly[904]: time="2026-10-25T18:47:33.506Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 diskcache=401a8a24-d4bc-4caf-9dc4-ccb84f2e32fb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-25T18:47:43.049" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.459" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1536gitlab # [ 189.373429] gitaly[904]: time="2026-10-25T18:47:33.506Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1581 command.oublock=32 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-25T18:47:43.049" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.459" grpc.time_ms=46.857 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1537gitlab # [ 189.381073] gitaly[904]: time="2026-10-25T18:47:33.516Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.105" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.516" pid=904 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1538gitlab # [ 189.381915] gitaly[904]: time="2026-10-25T18:47:33.517Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.105" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.516" grpc.time_ms=0.698 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1539gitlab # [ 189.429091] gitaly[904]: time="2026-10-25T18:47:33.564Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-25T18:47:43.145" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.555" grpc.time_ms=8.676 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1540gitlab # [ 189.478457] gitaly[904]: time="2026-10-25T18:47:33.613Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:47:43.202" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.613" grpc.time_ms=0.226 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1541gitlab # [ 189.486423] gitaly[904]: time="2026-10-25T18:47:33.622Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-25T18:47:43.211" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:33.621" grpc.time_ms=0.298 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1542gitlab # [ 189.936759] gitaly[904]: time="2026-10-25T18:47:34.071Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67BYWXSCKJXVQ2FSS2TW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-25T18:47:44.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:34.060" grpc.time_ms=10.637 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1543gitlab # [ 190.208456] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:34 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 17701544gitlab # [ 190.213148] setup-gitlab-start[2464]: {"id":1,"message":null,"error":null}1545gitlab # [ 190.926238] sidekiq-cluster[2447]: {"severity":"INFO","time":"2026-10-25T18:47:35.044Z","message":"Starting cluster with 1 processes"}1546gitlab # [ 195.551086] gitaly[904]: time="2026-10-25T18:47:39.685Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67JKF8SBMAXHJ2N7P2WHS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-25T18:47:49.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:39.679" grpc.time_ms=6.272 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1547gitlab # [ 196.019969] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:40 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 7791548gitlab # [ 196.025711] setup-gitlab-start[2178]: + mkdir -p /tmp/test-flake1549gitlab # [ 196.028213] setup-gitlab-start[2178]: + cd /tmp/test-flake1550gitlab # [ 196.029351] setup-gitlab-start[2178]: + git init -b master1551gitlab # [ 196.051355] setup-gitlab-start[2548]: Initialized empty Git repository in /tmp/test-flake/.git/1552gitlab # [ 196.052461] setup-gitlab-start[2178]: + git config user.name test1553gitlab # [ 196.055342] setup-gitlab-start[2178]: + git config user.email test@example.com1554gitlab # [ 196.058434] setup-gitlab-start[2178]: + cat1555gitlab # [ 196.060855] setup-gitlab-start[2178]: + git add flake.nix1556gitlab # [ 196.064110] setup-gitlab-start[2178]: + git commit -m 'initial commit'1557gitlab # [ 196.086573] setup-gitlab-start[2553]: [master (root-commit) 7c9d756] initial commit1558gitlab # [ 196.087687] setup-gitlab-start[2553]: 1 file changed, 10 insertions(+)1559gitlab # [ 196.089411] setup-gitlab-start[2553]: create mode 100644 flake.nix1560gitlab # [ 196.090520] setup-gitlab-start[2178]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1561gitlab # [ 196.096906] setup-gitlab-start[2178]: + git push -u origin master1562gitlab # [ 196.169578] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:40 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 461563gitlab # [ 196.289955] gitaly[904]: time="2026-10-25T18:47:40.425Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KGQKNJNQN4F2GX885DX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:40.412" grpc.time_ms=13.191 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1564gitlab # [ 196.302755] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:40 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1271565gitlab # [ 196.576507] gitaly[904]: time="2026-10-25T18:47:40.711Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-25T18:47:50.029" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:40.700" grpc.time_ms=10.938 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1566gitlab # [ 196.634156] gitaly[904]: time="2026-10-25T18:47:40.769Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-25T18:48:10.058" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-25T18:47:40.758" grpc.time_ms=11.158 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1567gitlab # [ 196.667154] gitaly[904]: time="2026-10-25T18:47:40.802Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS duration_ms=198ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/allowed"1568gitlab # [ 196.693910] gitaly[904]: time="2026-10-25T18:47:40.829Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS duration_ms=25ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/pre_receive"1569gitlab # [ 196.694896] gitaly[904]: time="2026-10-25T18:47:40.830Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:47:40.601" grpc.time_ms=229.136 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1570gitlab # [ 196.755467] gitaly[904]: time="2026-10-25T18:47:40.891Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:47:40.889" grpc.time_ms=1.115 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1571gitlab # [ 196.808858] gitaly[904]: time="2026-10-25T18:47:40.944Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:47:40.943" grpc.time_ms=0.675 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1572gitlab # [ 196.849531] gitaly[904]: time="2026-10-25T18:47:40.984Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:47:40.984" grpc.time_ms=0.567 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1573gitlab # [ 196.933846] gitaly[904]: time="2026-10-25T18:47:41.069Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-25T18:47:51.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:41.059" grpc.time_ms=9.296 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1574gitlab # [ 196.956602] gitaly[904]: time="2026-10-25T18:47:41.092Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1567 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-25T18:47:51.025" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:47:41.075" grpc.time_ms=16.276 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1575gitlab # [ 196.986641] gitaly[904]: time="2026-10-25T18:47:41.122Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS duration_ms=105ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/post_receive"1576gitlab # [ 196.987725] gitaly[904]: time="2026-10-25T18:47:41.122Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:47:41.015" grpc.time_ms=106.647 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1577gitlab # [ 197.011133] gitaly[904]: time="2026-10-25T18:47:41.146Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS diskcache=b055ddc5-22c0-4c3c-acea-c831745286a9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:40.528" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1578gitlab # [ 197.012448] gitaly[904]: time="2026-10-25T18:47:41.146Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=225 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13919 command.oublock=32 command.real_time_ms=608 command.spawn_token_fork_ms=0 command.system_time_ms=142 command.user_time_ms=83 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:40.528" grpc.time_ms=618.349 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1579gitlab # [ 197.013294] gitlab-workhorse[829]: time="2026-10-25T18:47:41Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521580gitlab # [ 197.013986] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:41 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 6931581gitlab # [ 197.037606] setup-gitlab-start[2557]: To http://gitlab/root/test-flake.git1582gitlab # [ 197.038710] setup-gitlab-start[2557]: * [new branch] master -> master1583gitlab # [ 197.041772] setup-gitlab-start[2557]: branch 'master' set up to track 'origin/master'.1584gitlab # [ 197.044310] systemd[1]: Finished setup-gitlab.service.1585gitlab # [ 197.046915] systemd[1]: Starting nixbot CI service...1586gitlab: (finished: waiting for unit setup-gitlab.service, in 198.05 seconds)1587gitlab: waiting for unit nixbot.service1588gitlab # [ 198.334914] nixbot[2636]: {"ts": "2026-10-25T18:47:42.469Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1589gitlab # [ 198.368657] nixbot[2636]: {"ts": "2026-10-25T18:47:42.504Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1590gitlab # [ 198.396975] nixbot[2636]: {"ts": "2026-10-25T18:47:42.532Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1591gitlab # [ 198.406745] nixbot[2636]: {"ts": "2026-10-25T18:47:42.542Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1592gitlab # [ 198.415257] nixbot[2636]: {"ts": "2026-10-25T18:47:42.550Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1593gitlab # [ 198.427599] nixbot[2636]: {"ts": "2026-10-25T18:47:42.563Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1594gitlab # [ 198.433624] nixbot[2636]: {"ts": "2026-10-25T18:47:42.569Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1595gitlab # [ 198.440067] nixbot[2636]: {"ts": "2026-10-25T18:47:42.575Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1596gitlab # [ 198.452730] nixbot[2636]: {"ts": "2026-10-25T18:47:42.588Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1597gitlab # [ 198.457749] nixbot[2636]: {"ts": "2026-10-25T18:47:42.593Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1598gitlab # [ 198.467062] nixbot[2636]: {"ts": "2026-10-25T18:47:42.602Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1599gitlab # [ 198.480398] nixbot[2636]: {"ts": "2026-10-25T18:47:42.615Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1600gitlab # [ 198.488401] nixbot[2636]: {"ts": "2026-10-25T18:47:42.623Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1601gitlab # [ 198.495442] nixbot[2636]: {"ts": "2026-10-25T18:47:42.630Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1602gitlab # [ 198.504731] nixbot[2636]: {"ts": "2026-10-25T18:47:42.640Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1603gitlab # [ 198.511481] nixbot[2636]: {"ts": "2026-10-25T18:47:42.647Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1604gitlab # [ 198.518466] nixbot[2636]: {"ts": "2026-10-25T18:47:42.653Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1605gitlab # [ 198.524366] nixbot[2636]: {"ts": "2026-10-25T18:47:42.659Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1606gitlab # [ 198.532352] nixbot[2636]: {"ts": "2026-10-25T18:47:42.667Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1607gitlab # [ 198.536080] nixbot[2636]: {"ts": "2026-10-25T18:47:42.671Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1608gitlab # [ 198.539924] nixbot[2636]: {"ts": "2026-10-25T18:47:42.675Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1609gitlab # [ 198.548838] nixbot[2636]: {"ts": "2026-10-25T18:47:42.684Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1610gitlab # [ 198.553386] nixbot[2636]: {"ts": "2026-10-25T18:47:42.689Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1611gitlab # [ 198.559207] nixbot[2636]: {"ts": "2026-10-25T18:47:42.694Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1612gitlab # [ 198.562317] nixbot[2636]: {"ts": "2026-10-25T18:47:42.697Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1613gitlab # [ 198.568211] nixbot[2636]: {"ts": "2026-10-25T18:47:42.703Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1614gitlab # [ 198.571664] nixbot[2636]: {"ts": "2026-10-25T18:47:42.707Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1615gitlab # [ 198.576381] nixbot[2636]: {"ts": "2026-10-25T18:47:42.711Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1616gitlab # [ 198.580892] nixbot[2636]: {"ts": "2026-10-25T18:47:42.716Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1617gitlab # [ 198.599965] nixbot[2636]: {"ts": "2026-10-25T18:47:42.735Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1618gitlab # [ 198.607139] nixbot[2636]: {"ts": "2026-10-25T18:47:42.742Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1619gitlab # [ 198.614138] nixbot[2636]: {"ts": "2026-10-25T18:47:42.749Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1620gitlab # [ 198.620532] nixbot[2636]: {"ts": "2026-10-25T18:47:42.756Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1621gitlab # [ 198.631351] nixbot[2636]: {"ts": "2026-10-25T18:47:42.766Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1622gitlab # [ 198.635305] nixbot[2636]: {"ts": "2026-10-25T18:47:42.770Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1623gitlab # [ 198.643375] nixbot[2636]: {"ts": "2026-10-25T18:47:42.778Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1624gitlab # [ 198.651879] nixbot[2636]: {"ts": "2026-10-25T18:47:42.787Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1625gitlab # [ 198.661561] nixbot[2636]: {"ts": "2026-10-25T18:47:42.797Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"}1626gitlab # [ 198.670436] nixbot[2636]: {"ts": "2026-10-25T18:47:42.805Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"}1627gitlab # [ 199.069758] nixbot[2636]: {"ts": "2026-10-25T18:47:43.205Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [2636]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}1628gitlab # [ 199.072510] nixbot[2636]: {"ts": "2026-10-25T18:47:43.205Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"}1629gitlab # [ 199.092850] nixbot[2636]: {"ts": "2026-10-25T18:47:43.228Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37892 - \"GET /health HTTP/1.1\" 200"}1630gitlab # [ 199.099654] systemd[1]: Started nixbot CI service.1631gitlab # [ 199.102611] systemd[1]: Reached target Multi-User System.1632gitlab # [ 199.105956] systemd[1]: Startup finished in 1.315s (kernel) + 3.603s (initrd) + 3min 14.186s (userspace) = 3min 19.105s.1633gitlab # [ 199.300289] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:43 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2061634gitlab # [ 199.321778] nixbot[2636]: {"ts": "2026-10-25T18:47:43.457Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1635gitlab # [ 199.374252] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 381636gitlab # [ 199.382172] nixbot[2636]: {"ts": "2026-10-25T18:47:43.517Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1637gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds)1638gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1639gitlab # [ 199.503925] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 1141640gitlab # [ 199.532123] nixbot[2636]: {"ts": "2026-10-25T18:47:43.667Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37894 - \"GET /health HTTP/1.1\" 200"}1641gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds)1642subtest: project discovered and webhook registered1643gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1644gitlab # [ 199.644644] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 771645gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1646(finished: subtest: project discovered and webhook registered, in 0.11 seconds)1647subtest: push delivers the webhook1648gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1649gitlab # [ 199.698082] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:43 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1781650gitlab # [ 199.743854] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 371651gitlab # [ 199.798091] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:43 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 431652gitlab # [ 199.852505] gitaly[904]: time="2026-10-25T18:47:43.987Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67Q2275KCSA7D51GN52EX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-25T18:48:13.072" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:47:43.973" grpc.time_ms=14.016 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1653gitlab # [ 199.885427] gitaly[904]: time="2026-10-25T18:47:44.021Z" level=info msg="finished unary call with code OK" catfile.duration_ms=23 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=23 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67Q2275KCSA7D51GN52EX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:13.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:47:43.996" grpc.time_ms=24.228 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1654gitlab # [ 199.914995] gitaly[904]: time="2026-10-25T18:47:44.050Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67Q2275KCSA7D51GN52EX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-25T18:47:54.020" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:47:44.041" grpc.time_ms=9.332 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1655gitlab # [ 199.937626] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1341656gitlab # [ 199.996260] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 551657gitlab # [ 199.999810] nixbot[2636]: {"ts": "2026-10-25T18:47:44.135Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1658gitlab # [ 200.109492] gitaly[904]: time="2026-10-25T18:47:44.245Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67QA5RFE5AC0K4DYHD3FC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.231" grpc.time_ms=13.75 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1659gitlab # [ 200.138600] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 761660gitlab # [ 200.241573] gitaly[904]: time="2026-10-25T18:47:44.377Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QCKGPQ1V870FV2WDNPP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.365" pid=904 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1661gitlab # [ 200.264387] gitaly[904]: time="2026-10-25T18:47:44.377Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QCKGPQ1V870FV2WDNPP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.365" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1662gitlab # [ 200.290347] gitaly[904]: time="2026-10-25T18:47:44.377Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QCKGPQ1V870FV2WDNPP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.365" grpc.time_ms=12.362 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1663gitlab # [ 200.307478] gitlab-workhorse[829]: time="2026-10-25T18:47:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131664gitlab # [ 200.309591] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1661665gitlab # [ 200.368721] gitaly[904]: time="2026-10-25T18:47:44.504Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QHYVR3QF417W6JNDE97 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.483" pid=904 remote_ip="2001:db8:1::1" request_sha=51f85d96ee7f97f1ba620845f52b10c1a52cd7e9 response_bytes=229 span.kind=server system=grpc1666gitlab # [ 200.388904] gitaly[904]: time="2026-10-25T18:47:44.504Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QHYVR3QF417W6JNDE97 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.483" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1667gitlab # [ 200.407328] gitaly[904]: time="2026-10-25T18:47:44.542Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QHYVR3QF417W6JNDE97 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.483" grpc.time_ms=59.334 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1668gitlab # [ 200.408061] gitlab-workhorse[829]: time="2026-10-25T18:47:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2291669gitlab # [ 200.408643] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 229 "" "git/2.55.0" 961670gitlab # [ 200.501257] gitaly[904]: time="2026-10-25T18:47:44.636Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67QPKEB1GGSW9F89BNTEY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.627" grpc.time_ms=9.34 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1671gitlab # [ 200.523357] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 631672gitlab # [ 200.573341] gitaly[904]: time="2026-10-25T18:47:44.708Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QRPVW67ANW1QD9KQ66N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.697" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1673gitlab # [ 200.594037] gitaly[904]: time="2026-10-25T18:47:44.709Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QRPVW67ANW1QD9KQ66N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.697" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1674gitlab # [ 200.613876] gitaly[904]: time="2026-10-25T18:47:44.749Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QRPVW67ANW1QD9KQ66N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.697" grpc.time_ms=52.401 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1675gitlab # [ 200.615124] gitlab-workhorse[829]: time="2026-10-25T18:47:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671676gitlab # [ 200.616190] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 871677gitlab # [ 200.714889] gitaly[904]: time="2026-10-25T18:47:44.850Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67QX1X0J5GH2E1K4GYJAV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.844" grpc.time_ms=6.103 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1678gitlab # [ 200.726933] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 501679gitlab # [ 200.772509] gitaly[904]: time="2026-10-25T18:47:44.908Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QYNVZ7T3DY84AWRW5P3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.895" pid=904 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1680gitlab # [ 200.773257] gitaly[904]: time="2026-10-25T18:47:44.908Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QYNVZ7T3DY84AWRW5P3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.895" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1681gitlab # [ 200.794366] gitaly[904]: time="2026-10-25T18:47:44.930Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67QYNVZ7T3DY84AWRW5P3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:47:44.895" grpc.time_ms=34.231 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1682gitlab # [ 200.795089] gitlab-workhorse[829]: time="2026-10-25T18:47:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091683gitlab # [ 200.795513] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 771684gitlab # [ 200.897614] gitaly[904]: time="2026-10-25T18:47:45.032Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:15.031" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:47:45.031" grpc.time_ms=0.762 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1685gitlab # [ 200.964134] gitaly[904]: time="2026-10-25T18:47:45.099Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-25T18:48:15.086" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:47:45.087" grpc.time_ms=11.921 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1686gitlab # [ 200.986676] gitaly[904]: time="2026-10-25T18:47:45.122Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:47:55.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:47:45.115" grpc.time_ms=6.291 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1687gitlab # [ 201.027122] gitaly[904]: time="2026-10-25T18:47:45.162Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:47:55.043" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:47:45.153" grpc.time_ms=8.758 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688gitlab # [ 201.091101] gitaly[904]: time="2026-10-25T18:47:45.226Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-25T18:47:55.115" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:45.226" grpc.time_ms=0.371 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689gitlab # [ 201.129887] gitaly[904]: time="2026-10-25T18:47:45.265Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-25T18:47:55.145" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:47:45.256" grpc.time_ms=9.146 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1690gitlab # [ 201.206305] gitaly[904]: time="2026-10-25T18:47:45.341Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:47:45.341" grpc.time_ms=0.44 pid=904 span.kind=server system=grpc1691gitlab # [ 201.257317] gitaly[904]: time="2026-10-25T18:47:45.392Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:47:45.392" grpc.time_ms=0.569 pid=904 span.kind=server system=grpc1692gitlab # [ 201.284627] gitaly[904]: time="2026-10-25T18:47:45.420Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 diskcache=ace98d4c-dbe1-442d-9c8f-c781f98b04e4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:47:55.180" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:45.291" pid=904 ref=refs/keep-around/7c9d7561d2f49577267b5e147c6e6f8726013972 remote_ip="2001:db8:1::1" revision=7c9d7561d2f49577267b5e147c6e6f8726013972 span.kind=server system=grpc username=root1693gitlab # [ 201.285954] gitaly[904]: time="2026-10-25T18:47:45.420Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=102 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5432 command.oublock=8 command.real_time_ms=120 command.spawn_token_fork_ms=0 command.system_time_ms=71 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:47:55.180" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:47:45.291" grpc.time_ms=128.811 pid=904 ref=refs/keep-around/7c9d7561d2f49577267b5e147c6e6f8726013972 remote_ip="2001:db8:1::1" revision=7c9d7561d2f49577267b5e147c6e6f8726013972 span.kind=server system=grpc username=root1694gitlab # [ 201.312520] gitaly[904]: time="2026-10-25T18:47:45.448Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:15.447" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:47:45.447" grpc.time_ms=0.363 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1695gitlab # [ 201.469485] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7c9d7561d2f49577267b5e147c6e6f8726013972 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 6241696gitlab # [ 201.491867] nixbot[2636]: {"ts": "2026-10-25T18:47:45.627Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-7c9d7561d2f4-0"}1697gitlab # [ 201.672131] nixbot[2636]: {"ts": "2026-10-25T18:47:45.807Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-7c9d7561d2f4-0", "argv0": "bwrap"}1698gitlab # [ 201.763196] systemd[1]: Started Nix Daemon.1699gitlab # [ 201.828132] nix-daemon[2801]: accepted connection from pid 2793, user nixbot1700gitlab # [ 206.042391] nix-daemon[2801]: accepted connection from pid 2843, user nixbot1701gitlab # [ 206.059720] nix-daemon[2801]: accepted connection from pid 2793, user nixbot1702gitlab # [ 210.336759] nix-daemon[2801]: accepted connection from pid 2793, user nixbot1703gitlab # [ 210.434294] nix-daemon[2801]: accepted connection from pid 2891, user nixbot1704gitlab # [ 210.478097] gitaly[904]: time="2026-10-25T18:47:54.613Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T681DTKQVJ4MG8D33JP7FG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:24.611" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:47:54.612" grpc.time_ms=0.824 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1705gitlab # [ 210.557837] gitaly[904]: time="2026-10-25T18:47:54.693Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M5T681DTKQVJ4MG8D33JP7FG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:04.012" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:47:54.683" grpc.time_ms=10.179 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1706gitlab # [ 210.593631] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:54 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7c9d7561d2f49577267b5e147c6e6f8726013972 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1741707gitlab # [ 210.662631] gitaly[904]: time="2026-10-25T18:47:54.798Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T681KM46M08J9VJEZ5XPY3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:24.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:47:54.797" grpc.time_ms=0.685 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1708gitlab # [ 210.732608] gitaly[904]: time="2026-10-25T18:47:54.868Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T681KM46M08J9VJEZ5XPY3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:04.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:47:54.859" grpc.time_ms=8.13 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1709gitlab # [ 210.811885] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:54 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7c9d7561d2f49577267b5e147c6e6f8726013972 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2061710gitlab # [ 215.205383] sidekiq-cluster[2508]: 2026-10-25T18:47:59.340Z pid=2508 tid=xus INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2508, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>436514816}1711gitlab # [ 215.369188] sidekiq-cluster[2508]: W, [2026-10-25T18:47:59.504669 #2508] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1712gitlab # [ 215.432835] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:47:59.568Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1713gitlab # [ 215.562780] nix-daemon[2801]: accepted connection from pid 2967, user nixbot1714gitlab # [ 215.648433] gitaly[904]: time="2026-10-25T18:47:59.783Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T686FEVB5W4APYSTA586BQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:29.081" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:47:59.782" grpc.time_ms=0.822 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1715gitlab # [ 215.689312] gitaly[904]: time="2026-10-25T18:47:59.824Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T686FEVB5W4APYSTA586BQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:09.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:47:59.813" grpc.time_ms=11.363 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1716gitlab # [ 215.730625] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:47:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7c9d7561d2f49577267b5e147c6e6f8726013972 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1391717gitlab # [ 215.906731] nixbot[2636]: {"ts": "2026-10-25T18:48:00.041Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "7c9d7561d2f49577267b5e147c6e6f8726013972", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}1718gitlab # [ 216.003168] gitaly[904]: time="2026-10-25T18:48:00.137Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M5T686TES7XE2HSVXSW7D1Y4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:00.130" grpc.time_ms=7.546 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1719gitlab # [ 216.032136] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 601720gitlab # [ 216.048199] gitaly[904]: time="2026-10-25T18:48:00.183Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T686WCE037V2A70ZGQC5CC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:00.174" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1721gitlab # [ 216.065758] gitaly[904]: time="2026-10-25T18:48:00.201Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T686WCE037V2A70ZGQC5CC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:00.174" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1722gitlab # [ 216.078163] gitaly[904]: time="2026-10-25T18:48:00.201Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T686WCE037V2A70ZGQC5CC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:00.174" grpc.time_ms=26.858 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1723gitlab # [ 216.090949] gitlab-workhorse[829]: time="2026-10-25T18:48:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671724gitlab # [ 216.092745] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 611725gitlab # [ 216.147733] gitaly[904]: time="2026-10-25T18:48:00.283Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T686ZQ595S5ZP66WJSGDA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:00.278" grpc.time_ms=5.059 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1726gitlab # [ 216.160344] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 351727gitlab # [ 216.194673] gitaly[904]: time="2026-10-25T18:48:00.330Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T6870XFJK9R8DNS5KX8N4G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:00.323" pid=904 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1728gitlab # [ 216.204037] gitaly[904]: time="2026-10-25T18:48:00.330Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T6870XFJK9R8DNS5KX8N4G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:00.323" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1729gitlab # [ 216.216551] gitlab-workhorse[829]: time="2026-10-25T18:48:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671730gitlab # [ 216.218285] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 451731gitlab # [ 216.220110] gitaly[904]: time="2026-10-25T18:48:00.330Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T6870XFJK9R8DNS5KX8N4G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:00.323" grpc.time_ms=6.81 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1732gitlab # [ 217.054089] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:01.188Z","message":"GitLab reliable fetch activated!"}1733gitlab # [ 217.057838] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:01.189Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1734gitlab # [ 235.586105] postgres[3172]: [3172] ERROR: relation "table_without_models" does not exist1735gitlab # [ 235.589194] postgres[3172]: [3172] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1736gitlab # [ 237.935745] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:22.071Z","message":"Booted Rails 7.2.3.1 application in production environment"}1737gitlab # [ 237.940152] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:22.075Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1738gitlab # [ 237.944355] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:22.075Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1739gitlab # [ 237.945090] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:22.080Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1740gitlab # [ 237.954598] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:22.090Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1741gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 38.35 seconds)1742gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1743gitlab # [ 238.095127] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:22 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 361744gitlab # [ 238.199396] gitaly[904]: time="2026-10-25T18:48:22.333Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WEXEQJSF90K71ETDWDE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:22.320" grpc.time_ms=12.218 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1745gitlab # [ 238.201663] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:22 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 961746gitlab # Enumerating objects: 5, done.1747gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1748gitlab # Delta compression using up to 4 threads1749gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1750gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done.1751gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1752gitlab # [ 238.487515] gitaly[904]: time="2026-10-25T18:48:22.623Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-25T18:48:52.614" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-25T18:48:22.615" grpc.time_ms=7.551 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753gitlab # [ 238.517200] gitaly[904]: time="2026-10-25T18:48:22.652Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M5T68WK4HT18GVJXPSFFRETH duration_ms=89ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/allowed"1754gitlab # [ 238.560729] gitaly[904]: time="2026-10-25T18:48:22.696Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M5T68WK4HT18GVJXPSFFRETH duration_ms=42ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/pre_receive"1755gitlab # [ 238.563078] gitaly[904]: time="2026-10-25T18:48:22.698Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:22.562" grpc.time_ms=136.374 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1756gitlab # [ 238.627058] gitaly[904]: time="2026-10-25T18:48:22.762Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:22.761" grpc.time_ms=0.761 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1757gitlab # [ 238.687379] gitaly[904]: time="2026-10-25T18:48:22.822Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:22.822" grpc.time_ms=0.502 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1758gitlab # [ 238.741533] gitaly[904]: time="2026-10-25T18:48:22.876Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:22.874" grpc.time_ms=2.301 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1759gitlab # [ 238.857960] gitaly[904]: time="2026-10-25T18:48:22.993Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M5T68WK4HT18GVJXPSFFRETH duration_ms=72ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/post_receive"1760gitlab # [ 238.858868] gitaly[904]: time="2026-10-25T18:48:22.994Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:22.920" grpc.time_ms=73.898 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1761gitlab # [ 238.896193] gitaly[904]: time="2026-10-25T18:48:23.031Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH diskcache=db3aed25-5ec1-43d7-a215-11fa51cb059f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:22.476" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1762gitlab # To http://gitlab/root/test-flake.git1763gitlab # 7c9d756..a786a2e [ 238.913939] gitaly[904]: time="2026-10-25T18:48:23.032Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=254 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=901120 command.minflt=13488 command.oublock=32 command.real_time_ms=546 command.spawn_token_fork_ms=0 command.system_time_ms=149 command.user_time_ms=105 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:22.476" grpc.time_ms=555.88 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1764gitlab # master -> master1765gitlab # [ 238.915330] gitlab-workhorse[829]: time="2026-10-25T18:48:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521766gitlab # [ 238.916491] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:23 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 6601767gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.93 seconds)1768gitlab: must succeed: git -C /tmp/test-flake rev-parse master1769gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1770gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2'1771gitlab # [ 239.007461] nixbot[2636]: {"ts": "2026-10-25T18:48:23.142Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:45978 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1\" 200"}1772gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2', in 0.07 seconds)1773gitlab # [ 239.232518] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:23.366Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3bf4e322c4c318db957f0830","created_at":"2026-10-25T18:47:33.379Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:33.405Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3bf4e322c4c318db957f0830: start","job_status":"start","queue_duration_s":49.961681,"scheduling_latency_s":49.961752}1774gitlab # [ 239.464390] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:23.599Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3bf4e322c4c318db957f0830","created_at":"2026-10-25T18:47:33.379Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:33.405Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3bf4e322c4c318db957f0830: done: 0.23317 sec","job_status":"done","queue_duration_s":49.961681,"scheduling_latency_s":49.961752,"redis_calls":23,"redis_duration_s":0.007462,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003959,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00208,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09614993,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.23317,"completed_at":"2026-10-25T18:48:23.599Z","load_balancing_strategy":"primary","db_duration_s":0.061075,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1775gitlab # [ 239.544677] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:23.679Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"0da5389a20e0f89da129cebe","created_at":"2026-10-25T18:47:33.412Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:33.415Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-0da5389a20e0f89da129cebe: start","job_status":"start","queue_duration_s":50.263901,"scheduling_latency_s":50.263945}1776gitlab # [ 239.659743] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:23.795Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"0da5389a20e0f89da129cebe","created_at":"2026-10-25T18:47:33.412Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:33.415Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-0da5389a20e0f89da129cebe: done: 0.1153 sec","job_status":"done","queue_duration_s":50.263901,"scheduling_latency_s":50.263945,"redis_calls":14,"redis_duration_s":0.003873,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000917,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001757,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001199,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.086064011,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.1153,"completed_at":"2026-10-25T18:48:23.795Z","load_balancing_strategy":"primary","db_duration_s":0.025123,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1777gitlab # [ 239.734638] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:23.869Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"cb3284a714de519f075c42b8","created_at":"2026-10-25T18:47:33.988Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:33.993Z","job_size_bytes":5,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-cb3284a714de519f075c42b8: start","job_status":"start","queue_duration_s":49.876552,"scheduling_latency_s":49.876603}1778gitlab # [ 239.897838] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:24.033Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"cb3284a714de519f075c42b8","created_at":"2026-10-25T18:47:33.988Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:33.993Z","job_size_bytes":5,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-cb3284a714de519f075c42b8: done: 0.163716 sec","job_status":"done","queue_duration_s":49.876552,"scheduling_latency_s":49.876603,"redis_calls":12,"redis_duration_s":0.003026,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000883,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001728,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000320991,"duration_s":0.163716,"completed_at":"2026-10-25T18:48:24.033Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028184,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1779gitlab # [ 239.977644] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:24.112Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"e88ad4bff9d2701d2a018d90","created_at":"2026-10-25T18:47:34.014Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:34.017Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-e88ad4bff9d2701d2a018d90: start","job_status":"start","queue_duration_s":50.095182,"scheduling_latency_s":50.095231}1780gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2'1781gitlab # [ 240.052791] nixbot[2636]: {"ts": "2026-10-25T18:48:24.187Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:45992 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1\" 200"}1782gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2', in 0.04 seconds)1783gitlab # [ 240.254692] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:24.388Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"e88ad4bff9d2701d2a018d90","created_at":"2026-10-25T18:47:34.014Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:34.017Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-e88ad4bff9d2701d2a018d90: done: 0.276188 sec","job_status":"done","queue_duration_s":50.095182,"scheduling_latency_s":50.095231,"redis_calls":11,"redis_duration_s":0.003072,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000735,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001812,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000525,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.213,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.213,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.175439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000578844,"duration_s":0.276188,"completed_at":"2026-10-25T18:48:24.388Z","load_balancing_strategy":"primary","db_duration_s":0.086924,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1784gitlab # [ 240.256796] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:24.391Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"801aff92b1e22934c2b2bc35","created_at":"2026-10-25T18:47:34.019Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:34.022Z","job_size_bytes":89,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-801aff92b1e22934c2b2bc35: start","job_status":"start","queue_duration_s":50.369019,"scheduling_latency_s":50.369052}1785gitlab # [ 240.336312] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:24.439Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"801aff92b1e22934c2b2bc35","created_at":"2026-10-25T18:47:34.019Z","trace_propagation_headers":{"sentry-trace":"f8f263e7829f49a6aceece55487ec280-23b8d0806123429e","baggage":"sentry-trace_id=f8f263e7829f49a6aceece55487ec280,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67BYWXSCKJXVQ2FSS2TW3","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:34.022Z","job_size_bytes":89,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-801aff92b1e22934c2b2bc35: done: 0.047664 sec","job_status":"done","queue_duration_s":50.369019,"scheduling_latency_s":50.369052,"redis_calls":11,"redis_duration_s":0.003268,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000977,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000569,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000678857,"duration_s":0.047664,"completed_at":"2026-10-25T18:48:24.439Z","load_balancing_strategy":"primary","db_duration_s":0.006283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1786gitlab # [ 240.382652] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:24.441Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 7c9d7561d2f49577267b5e147c6e6f8726013972 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f7076ed8d5c6b6200529b069","created_at":"2026-10-25T18:47:41.048Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-fb35a529ee964698","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:41.051Z","job_size_bytes":185,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f7076ed8d5c6b6200529b069: start","job_status":"start","queue_duration_s":43.389681,"scheduling_latency_s":43.389706}1787gitlab # [ 240.383483] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:24.453Z","message":"changes: \"0000000000000000000000000000000000000000 7c9d7561d2f49577267b5e147c6e6f8726013972 refs/heads/master\\n\""}1788gitlab # [ 240.384931] gitaly[904]: time="2026-10-25T18:48:24.487Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:34.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:24.478" grpc.time_ms=9.082 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789gitlab # [ 240.385634] gitaly[904]: time="2026-10-25T18:48:24.502Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-25T18:48:34.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:24.491" grpc.time_ms=10.425 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790gitlab # [ 240.433084] gitaly[904]: time="2026-10-25T18:48:24.568Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=24 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-25T18:48:34.066" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:24.537" grpc.time_ms=31.402 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1791gitlab # [ 240.470543] gitaly[904]: time="2026-10-25T18:48:24.606Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:24.605" grpc.time_ms=0.638 pid=904 span.kind=server system=grpc1792gitlab # [ 240.502720] gitaly[904]: time="2026-10-25T18:48:24.638Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:24.637" grpc.time_ms=0.647 pid=904 span.kind=server system=grpc1793gitlab # [ 240.519813] gitaly[904]: time="2026-10-25T18:48:24.655Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS diskcache=95287767-d132-4f90-9f11-cbb3945e243e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:48:34.106" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:24.577" pid=904 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1794gitlab # [ 240.520479] gitaly[904]: time="2026-10-25T18:48:24.655Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4821 command.oublock=8 command.real_time_ms=71 command.spawn_token_fork_ms=0 command.system_time_ms=40 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:48:34.106" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:24.577" grpc.time_ms=78.56 pid=904 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1795gitlab # [ 240.542637] gitaly[904]: time="2026-10-25T18:48:24.678Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-25T18:48:34.207" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:24.677" grpc.time_ms=0.186 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1796gitlab # [ 240.559356] gitaly[904]: time="2026-10-25T18:48:24.694Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=936 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-25T18:48:34.212" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:24.683" grpc.time_ms=11.483 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1797gitlab # [ 240.646520] gitaly[904]: time="2026-10-25T18:48:24.781Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-25T18:48:54.073" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:24.774" grpc.time_ms=7.7 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798gitlab # [ 240.678502] gitaly[904]: time="2026-10-25T18:48:24.814Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-25T18:48:34.013" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:24.813" grpc.time_ms=0.213 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799gitlab # [ 240.701986] gitaly[904]: time="2026-10-25T18:48:24.837Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-25T18:48:34.030" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-25T18:48:24.831" grpc.time_ms=6.584 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800gitlab # [ 240.722309] gitaly[904]: time="2026-10-25T18:48:24.857Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:34.046" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:24.846" grpc.time_ms=10.86 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab # [ 240.744869] gitaly[904]: time="2026-10-25T18:48:24.880Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:34.070" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:24.870" grpc.time_ms=9.823 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1802gitlab # [ 240.769329] gitaly[904]: time="2026-10-25T18:48:24.904Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=958 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-25T18:48:54.193" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:24.893" grpc.time_ms=11.123 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2'1804gitlab # [ 241.100777] nixbot[2636]: {"ts": "2026-10-25T18:48:25.235Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:46008 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1\" 200"}1805gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2', in 0.05 seconds)1806gitlab # [ 241.122676] gitaly[904]: time="2026-10-25T18:48:25.258Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-25T18:48:35.006" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:25.256" grpc.time_ms=1.549 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807gitlab # [ 241.195414] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:25.330Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 7c9d7561d2f49577267b5e147c6e6f8726013972 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f7076ed8d5c6b6200529b069","created_at":"2026-10-25T18:47:41.048Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-fb35a529ee964698","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:47:41.051Z","job_size_bytes":185,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f7076ed8d5c6b6200529b069: done: 0.889181 sec","job_status":"done","queue_duration_s":43.389681,"scheduling_latency_s":43.389706,"gitaly_calls":13,"gitaly_duration_s":0.298827,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.046003,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.004122,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006503,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001671,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003162,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.018334,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.012211,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.163,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.183,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.471076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004224,"exclusive_lock_hold_duration_s":0.028232079,"duration_s":0.889181,"completed_at":"2026-10-25T18:48:25.330Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.09811,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1808gitlab # [ 241.266447] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:25.401Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"adcffc0eae1f4ad5ef229116","created_at":"2026-10-25T18:47:45.467Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-4aa0d2632be44e6b","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:45.489Z","job_size_bytes":65,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-adcffc0eae1f4ad5ef229116: start","job_status":"start","queue_duration_s":39.911575,"scheduling_latency_s":39.911624}1809gitlab # [ 241.383702] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:25.518Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"adcffc0eae1f4ad5ef229116","created_at":"2026-10-25T18:47:45.467Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-4aa0d2632be44e6b","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:45.489Z","job_size_bytes":65,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-adcffc0eae1f4ad5ef229116: done: 0.117241 sec","job_status":"done","queue_duration_s":39.911575,"scheduling_latency_s":39.911624,"redis_calls":16,"redis_duration_s":0.0060290000000000005,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.0033,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000551467,"duration_s":0.117241,"completed_at":"2026-10-25T18:48:25.518Z","load_balancing_strategy":"primary","db_duration_s":0.02431,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1810gitlab # [ 241.432275] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:25.521Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0ad9c70c134e49ff24f9de4b","created_at":"2026-10-25T18:47:45.490Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-4aa0d2632be44e6b","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:45.494Z","job_size_bytes":65,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0ad9c70c134e49ff24f9de4b: start","job_status":"start","queue_duration_s":40.026789,"scheduling_latency_s":40.026823}1811gitlab # [ 241.489091] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:25.624Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0ad9c70c134e49ff24f9de4b","created_at":"2026-10-25T18:47:45.490Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-4aa0d2632be44e6b","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:45.494Z","job_size_bytes":65,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0ad9c70c134e49ff24f9de4b: done: 0.10304 sec","job_status":"done","queue_duration_s":40.026789,"scheduling_latency_s":40.026823,"redis_calls":21,"redis_duration_s":0.006268,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002768,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001921,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001579,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060734,"duration_s":0.10304,"completed_at":"2026-10-25T18:48:25.623Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029983,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1812gitlab # [ 241.491179] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:25.626Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"95a4ebc944228832a7afcab3","created_at":"2026-10-25T18:47:45.588Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-4aa0d2632be44e6b","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:45.592Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-95a4ebc944228832a7afcab3: start","job_status":"start","queue_duration_s":40.034149,"scheduling_latency_s":40.034182}1813gitlab # [ 241.693111] gitaly[904]: time="2026-10-25T18:48:25.827Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:25.827" grpc.time_ms=0.529 pid=904 span.kind=server system=grpc1814gitlab # [ 241.740169] gitaly[904]: time="2026-10-25T18:48:25.875Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:25.875" grpc.time_ms=0.477 pid=904 span.kind=server system=grpc1815gitlab # [ 241.777261] gitaly[904]: time="2026-10-25T18:48:25.912Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:25.912" grpc.time_ms=0.346 pid=904 span.kind=server system=grpc1816gitlab # [ 241.796498] gitaly[904]: time="2026-10-25T18:48:25.932Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 diskcache=06ddf8f8-ca1d-4f56-bc00-e45332c97b65 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-25T18:48:55.086" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:25.786" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab # [ 241.805425] gitaly[904]: time="2026-10-25T18:48:25.932Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=145 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7579 command.oublock=0 command.real_time_ms=138 command.spawn_token_fork_ms=0 command.system_time_ms=106 command.user_time_ms=39 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-25T18:48:55.086" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:25.786" grpc.time_ms=145.549 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818gitlab # [ 241.873953] gitaly[904]: time="2026-10-25T18:48:26.009Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-25T18:48:56.007" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.008" grpc.time_ms=1.106 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1819gitlab # [ 241.908741] gitaly[904]: time="2026-10-25T18:48:26.044Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67R2KYHXZSR4G1MCHJCN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:56.042" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.042" grpc.time_ms=1.056 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1820gitlab # [ 241.923202] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.058Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"95a4ebc944228832a7afcab3","created_at":"2026-10-25T18:47:45.588Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-4aa0d2632be44e6b","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:45.592Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2508,"job_status":"rescheduled","message":"PipelineProcessWorker JID-95a4ebc944228832a7afcab3: rescheduled"}1821gitlab # [ 241.934533] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.069Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"95a4ebc944228832a7afcab3","created_at":"2026-10-25T18:47:45.588Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-4aa0d2632be44e6b","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:47:45.592Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-95a4ebc944228832a7afcab3: done: 0.443232 sec","job_status":"done","queue_duration_s":40.034149,"scheduling_latency_s":40.034182,"gitaly_calls":3,"gitaly_duration_s":0.155299,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.024481,"redis_read_bytes":460,"redis_write_bytes":29298,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001089,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001054,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.009009,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.003738,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15585,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005036,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000563,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000502,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002401,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.017,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.217308,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.379557457,"duration_s":0.443232,"completed_at":"2026-10-25T18:48:26.069Z","load_balancing_strategy":"primary","db_duration_s":0.052992,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1822gitlab # [ 241.979634] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.071Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","7c9d7561d2f49577267b5e147c6e6f8726013972 a786a2e66da786487d047f8106fd2d599a2fa9a2 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"739415fc57bad6ce542f84ab","created_at":"2026-10-25T18:48:22.962Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-0f09133a292546e9","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:48:22.964Z","job_size_bytes":185,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-739415fc57bad6ce542f84ab: start","job_status":"start","queue_duration_s":3.107046,"scheduling_latency_s":3.107073}1823gitlab # [ 241.980611] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.084Z","message":"changes: \"7c9d7561d2f49577267b5e147c6e6f8726013972 a786a2e66da786487d047f8106fd2d599a2fa9a2 refs/heads/master\\n\""}1824gitlab # [ 241.981599] gitaly[904]: time="2026-10-25T18:48:26.110Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:36.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:26.101" grpc.time_ms=9.091 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1825gitlab # [ 241.983409] gitaly[904]: time="2026-10-25T18:48:26.119Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-25T18:48:36.023" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:26.114" grpc.time_ms=4.917 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1826gitlab # [ 241.991650] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.121Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"a62dffee3191b1de29a69f7e","created_at":"2026-10-25T18:48:26.120Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-f1f339439b7044dc","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"dfcac15dfea3b202f3e9f257","job_size_bytes":46,"pid":2508,"job_status":"deduplicated","message":"ProjectCacheWorker JID-a62dffee3191b1de29a69f7e: deduplicated: until executing","deduplication.type":"until executing"}1827gitlab # [ 242.039590] gitaly[904]: time="2026-10-25T18:48:26.136Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-25T18:48:36.040" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:26.130" grpc.time_ms=5.707 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1828gitlab # [ 242.040338] gitaly[904]: time="2026-10-25T18:48:26.148Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-25T18:48:56.142" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.142" grpc.time_ms=6.11 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1829gitlab # [ 242.041158] gitaly[904]: time="2026-10-25T18:48:26.159Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-25T18:48:36.065" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-25T18:48:26.155" grpc.time_ms=3.459 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1830gitlab # [ 242.043091] gitaly[904]: time="2026-10-25T18:48:26.173Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:36.074" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:26.164" grpc.time_ms=8.781 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1831gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2'1832gitlab # [ 242.107867] gitaly[904]: time="2026-10-25T18:48:26.243Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-25T18:48:36.152" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.242" grpc.time_ms=1.201 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1833gitlab # [ 242.130366] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.265Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","7c9d7561d2f49577267b5e147c6e6f8726013972 a786a2e66da786487d047f8106fd2d599a2fa9a2 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"739415fc57bad6ce542f84ab","created_at":"2026-10-25T18:48:22.962Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-0f09133a292546e9","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:48:22.964Z","job_size_bytes":185,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-739415fc57bad6ce542f84ab: done: 0.194254 sec","job_status":"done","queue_duration_s":3.107046,"scheduling_latency_s":3.107073,"gitaly_calls":7,"gitaly_duration_s":0.053638,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.017696,"redis_read_bytes":501,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.001816,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000739,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002896,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006443,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.005802,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.019,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.019,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000628292,"exclusive_lock_hold_duration_s":0.016393983,"duration_s":0.194254,"completed_at":"2026-10-25T18:48:26.265Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021071,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1834gitlab # [ 242.135063] nixbot[2636]: {"ts": "2026-10-25T18:48:26.269Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:46010 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1\" 200"}1835gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2', in 0.07 seconds)1836gitlab # [ 242.177607] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.313Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"dfcac15dfea3b202f3e9f257","created_at":"2026-10-25T18:48:24.503Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:24.516Z","job_size_bytes":46,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-dfcac15dfea3b202f3e9f257: start","job_status":"start","queue_duration_s":1.796793,"scheduling_latency_s":1.796832}1837gitlab # [ 242.207535] gitaly[904]: time="2026-10-25T18:48:26.343Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-26T00:48:26.340" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:26.340" grpc.time_ms=2.048 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 242.229199] gitaly[904]: time="2026-10-25T18:48:26.364Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-25T18:48:56.358" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.359" grpc.time_ms=5.632 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 242.260545] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.395Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"dfcac15dfea3b202f3e9f257","created_at":"2026-10-25T18:48:24.503Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:24.516Z","job_size_bytes":46,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-dfcac15dfea3b202f3e9f257: done: 0.082754 sec","job_status":"done","queue_duration_s":1.796793,"scheduling_latency_s":1.796832,"gitaly_calls":2,"gitaly_duration_s":0.020947,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.01546,"redis_read_bytes":69,"redis_write_bytes":4882,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000896,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000409,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.012809,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000209244,"duration_s":0.082754,"completed_at":"2026-10-25T18:48:26.395Z","load_balancing_strategy":"primary","db_duration_s":0.007929,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1840gitlab # [ 242.264495] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.396Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"483fe494dc3e178c913f098d","created_at":"2026-10-25T18:48:24.530Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:24.533Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-483fe494dc3e178c913f098d: start","job_status":"start","queue_duration_s":1.862851,"scheduling_latency_s":1.862866}1841gitlab # [ 242.301203] gitaly[904]: time="2026-10-25T18:48:26.436Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS diskcache=e765a7d4-50fc-4640-a2f3-c6562cf4d939 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-26T00:48:26.411" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.411" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842gitlab # [ 242.302155] gitaly[904]: time="2026-10-25T18:48:26.437Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1286 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-26T00:48:26.411" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.411" grpc.time_ms=25.305 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1843gitlab # [ 242.368224] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.502Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"483fe494dc3e178c913f098d","created_at":"2026-10-25T18:48:24.530Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:24.533Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-483fe494dc3e178c913f098d: done: 0.106147 sec","job_status":"done","queue_duration_s":1.862851,"scheduling_latency_s":1.862866,"gitaly_calls":1,"gitaly_duration_s":0.068718,"redis_calls":15,"redis_duration_s":0.00248,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000719,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000721,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000357,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000683,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.090887809,"duration_s":0.106147,"completed_at":"2026-10-25T18:48:26.502Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009604,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1844gitlab # [ 242.370786] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.503Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0c9ecff687209817ba85c4a5","created_at":"2026-10-25T18:48:24.671Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:24.674Z","job_size_bytes":23,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0c9ecff687209817ba85c4a5: start","job_status":"start","queue_duration_s":1.829463,"scheduling_latency_s":1.829477}1845gitlab # [ 242.429573] gitaly[904]: time="2026-10-25T18:48:26.565Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-25T18:48:56.558" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.559" grpc.time_ms=5.804 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1846gitlab # [ 242.443921] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.579Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0c9ecff687209817ba85c4a5","created_at":"2026-10-25T18:48:24.671Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:24.674Z","job_size_bytes":23,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0c9ecff687209817ba85c4a5: done: 0.075162 sec","job_status":"done","queue_duration_s":1.829463,"scheduling_latency_s":1.829477,"gitaly_calls":1,"gitaly_duration_s":0.047637,"redis_calls":16,"redis_duration_s":0.003032,"redis_read_bytes":52,"redis_write_bytes":2048,"redis_queues_calls":1,"redis_queues_duration_s":0.000289,"redis_queues_read_bytes":1,"redis_queues_write_bytes":979,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.00135,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075162,"completed_at":"2026-10-25T18:48:26.578Z","load_balancing_strategy":"primary","db_duration_s":0.001487,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1847gitlab # [ 242.445818] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:26.581Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"894f2a2aea3fa848193c9ca3","created_at":"2026-10-25T18:48:25.107Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8208aa223c6f96e2f5c3a278f31ae747512aa9092e8d24ee2a34988527643a80","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.114Z","job_size_bytes":467,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-894f2a2aea3fa848193c9ca3: start","job_status":"start","queue_duration_s":1.466178,"scheduling_latency_s":1.466219}1848gitlab # [ 242.479710] gitaly[904]: time="2026-10-25T18:48:26.615Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:56.614" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.614" grpc.time_ms=0.495 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1849gitlab # [ 242.522574] gitaly[904]: time="2026-10-25T18:48:26.658Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-25T18:48:36.009" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:26.650" grpc.time_ms=7.85 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1850gitlab # [ 242.611144] gitaly[904]: time="2026-10-25T18:48:26.746Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-25T18:48:56.745" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.745" grpc.time_ms=1.066 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1851gitlab # [ 242.639087] gitaly[904]: time="2026-10-25T18:48:26.755Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=33 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-25T18:48:56.752" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:26.752" grpc.time_ms=2.419 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1852gitlab # [ 242.743749] gitaly[904]: time="2026-10-25T18:48:26.878Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:36.226" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:26.866" grpc.time_ms=11.488 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1853gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2'1854gitlab # [ 243.171661] gitaly[904]: time="2026-10-25T18:48:27.307Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-25T18:48:57.289" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:27.289" grpc.time_ms=17.481 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1855gitlab # [ 243.224809] gitaly[904]: time="2026-10-25T18:48:27.360Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T67KN64FY0XE0NQ3YJF8NS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-25T18:48:57.343" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:27.343" grpc.time_ms=16.362 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1856gitlab # [ 243.245943] sidekiq-cluster[2508]: {"severity":"WARN","time":"2026-10-25T18:48:27.381Z","class":"CreatePipelineWorker","correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"7c9d7561d2f49577267b5e147c6e6f8726013972","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"7c9d7561d2f49577267b5e147c6e6f8726013972","ref":"refs/heads/master"},"retry":0}1857gitlab # [ 243.251073] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.386Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"894f2a2aea3fa848193c9ca3","created_at":"2026-10-25T18:48:25.107Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8208aa223c6f96e2f5c3a278f31ae747512aa9092e8d24ee2a34988527643a80","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.114Z","job_size_bytes":467,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-894f2a2aea3fa848193c9ca3: done: 0.805399 sec","job_status":"done","queue_duration_s":1.466178,"scheduling_latency_s":1.466219,"gitaly_calls":7,"gitaly_duration_s":0.157742,"redis_calls":51,"redis_duration_s":0.010381,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.00086,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004523,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002785,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.60786,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000289423,"duration_s":0.805399,"completed_at":"2026-10-25T18:48:27.386Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046712,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1858gitlab # [ 243.266151] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.387Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"63b5af1c-9435-431d-9bce-81fed6f3879b\"}"],"class":"WebHookWorker","jid":"d7d84477dcba74050127e7f3","created_at":"2026-10-25T18:48:25.223Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2e49531c6b05a05b618374b85426158017409228871f37e821f2a51ad2fa3dd5","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.230Z","job_size_bytes":1751,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d7d84477dcba74050127e7f3: start","job_status":"start","queue_duration_s":2.15774,"scheduling_latency_s":2.157754}1859gitlab # [ 243.326996] nixbot[2636]: {"ts": "2026-10-25T18:48:27.462Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40208 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1\" 200"}1860gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2', in 0.15 seconds)1861gitlab # [ 243.398181] nixbot[2636]: {"ts": "2026-10-25T18:48:27.533Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "0ff8d0e4-db61-4240-9ede-3e8bb6bcb46d", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "7c9d7561d2f49577267b5e147c6e6f8726013972", "pr": null}1862gitlab # [ 243.402897] nixbot[2636]: {"ts": "2026-10-25T18:48:27.538Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40214 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1863gitlab # [ 243.427367] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.561Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"63b5af1c-9435-431d-9bce-81fed6f3879b\"}"],"class":"WebHookWorker","jid":"d7d84477dcba74050127e7f3","created_at":"2026-10-25T18:48:25.223Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-44c7ae6874294bad","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2e49531c6b05a05b618374b85426158017409228871f37e821f2a51ad2fa3dd5","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.230Z","job_size_bytes":1751,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d7d84477dcba74050127e7f3: done: 0.174204 sec","job_status":"done","queue_duration_s":2.15774,"scheduling_latency_s":2.157754,"redis_calls":19,"redis_duration_s":0.003603,"redis_read_bytes":32,"redis_write_bytes":5366,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000882,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000317,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001544,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00086,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.0464701519999835,"cpu_s":0.0652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000274895,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.174204,"completed_at":"2026-10-25T18:48:27.561Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003612,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1864gitlab # [ 243.487509] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.563Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3856ce80da61bb9e0a93b48b","created_at":"2026-10-25T18:48:25.775Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.778Z","job_size_bytes":23,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3856ce80da61bb9e0a93b48b: start","job_status":"start","queue_duration_s":1.784767,"scheduling_latency_s":1.784782}1865gitlab # [ 243.531075] gitaly[904]: time="2026-10-25T18:48:27.666Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M5T691NF5JMZVG5PCNNQH8ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:27.655" grpc.time_ms=10.762 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1866gitlab # [ 243.533075] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:27 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 991867gitlab # [ 243.604200] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.739Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3856ce80da61bb9e0a93b48b","created_at":"2026-10-25T18:48:25.775Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.778Z","job_size_bytes":23,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3856ce80da61bb9e0a93b48b: done: 0.176163 sec","job_status":"done","queue_duration_s":1.784767,"scheduling_latency_s":1.784782,"redis_calls":10,"redis_duration_s":0.002852,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00064,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000245003,"duration_s":0.176163,"completed_at":"2026-10-25T18:48:27.739Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024679,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1868gitlab # [ 243.607624] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.743Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d5171339499671abd25eb7e4","created_at":"2026-10-25T18:48:25.936Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.940Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d5171339499671abd25eb7e4: start","job_status":"start","queue_duration_s":1.80273,"scheduling_latency_s":1.802746}1869gitlab # [ 243.661973] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.766Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d5171339499671abd25eb7e4","created_at":"2026-10-25T18:48:25.936Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.940Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d5171339499671abd25eb7e4: done: 0.023214 sec","job_status":"done","queue_duration_s":1.80273,"scheduling_latency_s":1.802746,"redis_calls":10,"redis_duration_s":0.001989,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001212,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025059,"duration_s":0.023214,"completed_at":"2026-10-25T18:48:27.766Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004981,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1870gitlab # [ 243.675104] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.767Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8ae64fedb19b765bcc608db1","created_at":"2026-10-25T18:48:25.951Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.956Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8ae64fedb19b765bcc608db1: start","job_status":"start","queue_duration_s":1.811236,"scheduling_latency_s":1.811249}1871gitlab # [ 243.716571] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.790Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8ae64fedb19b765bcc608db1","created_at":"2026-10-25T18:48:25.951Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.956Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8ae64fedb19b765bcc608db1: done: 0.023389 sec","job_status":"done","queue_duration_s":1.811236,"scheduling_latency_s":1.811249,"redis_calls":10,"redis_duration_s":0.002171,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000397,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000292495,"duration_s":0.023389,"completed_at":"2026-10-25T18:48:27.790Z","load_balancing_strategy":"primary","db_duration_s":0.005626,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1872gitlab # [ 243.728187] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b1f8fd9e2c676eeadef46d42","created_at":"2026-10-25T18:48:25.964Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.967Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b1f8fd9e2c676eeadef46d42: start","job_status":"start","queue_duration_s":1.824827,"scheduling_latency_s":1.824841}1873gitlab # [ 243.742797] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.816Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b1f8fd9e2c676eeadef46d42","created_at":"2026-10-25T18:48:25.964Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.967Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b1f8fd9e2c676eeadef46d42: done: 0.024553 sec","job_status":"done","queue_duration_s":1.824827,"scheduling_latency_s":1.824841,"redis_calls":13,"redis_duration_s":0.002542,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000547,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024668,"duration_s":0.024553,"completed_at":"2026-10-25T18:48:27.816Z","load_balancing_strategy":"primary","db_duration_s":0.005212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1874gitlab # [ 243.792502] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.817Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"78f179c068f355ef4ad48ca6","created_at":"2026-10-25T18:48:25.967Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.970Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-78f179c068f355ef4ad48ca6: start","job_status":"start","queue_duration_s":1.847668,"scheduling_latency_s":1.847682}1875gitlab # [ 243.797593] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.838Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"78f179c068f355ef4ad48ca6","created_at":"2026-10-25T18:48:25.967Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.970Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-78f179c068f355ef4ad48ca6: done: 0.020436 sec","job_status":"done","queue_duration_s":1.847668,"scheduling_latency_s":1.847682,"redis_calls":10,"redis_duration_s":0.001782,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025981,"duration_s":0.020436,"completed_at":"2026-10-25T18:48:27.838Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004001,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1876gitlab # [ 243.874740] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.839Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"498ebad55bf08e9073a11ed6","created_at":"2026-10-25T18:48:25.970Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.973Z","job_size_bytes":19,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-498ebad55bf08e9073a11ed6: start","job_status":"start","queue_duration_s":1.865879,"scheduling_latency_s":1.865892}1877gitlab # [ 243.875811] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.860Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"498ebad55bf08e9073a11ed6","created_at":"2026-10-25T18:48:25.970Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.973Z","job_size_bytes":19,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-498ebad55bf08e9073a11ed6: done: 0.021208 sec","job_status":"done","queue_duration_s":1.865879,"scheduling_latency_s":1.865892,"redis_calls":11,"redis_duration_s":0.0026089999999999998,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000421,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001462,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000726,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000276571,"duration_s":0.021208,"completed_at":"2026-10-25T18:48:27.860Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004651,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1878gitlab # [ 243.892783] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.861Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3854ad3e1a62445c0590fc20","created_at":"2026-10-25T18:48:25.975Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.977Z","job_size_bytes":66,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3854ad3e1a62445c0590fc20: start","job_status":"start","queue_duration_s":1.883896,"scheduling_latency_s":1.883909}1879gitlab # [ 243.908595] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.936Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3854ad3e1a62445c0590fc20","created_at":"2026-10-25T18:48:25.975Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.977Z","job_size_bytes":66,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3854ad3e1a62445c0590fc20: done: 0.074685 sec","job_status":"done","queue_duration_s":1.883896,"scheduling_latency_s":1.883909,"redis_calls":10,"redis_duration_s":0.002732,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001924,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000228521,"duration_s":0.074685,"completed_at":"2026-10-25T18:48:27.936Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.054596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1880gitlab # [ 243.983238] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.937Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c29b38bc39a29774349c37fb","created_at":"2026-10-25T18:48:25.978Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.980Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c29b38bc39a29774349c37fb: start","job_status":"start","queue_duration_s":1.956601,"scheduling_latency_s":1.956614}1881gitlab # [ 243.984659] gitaly[904]: time="2026-10-25T18:48:28.086Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T691RM706FZ3C3M2VFHA77 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.061" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1882gitlab # [ 243.985965] gitlab-workhorse[829]: time="2026-10-25T18:48:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671883gitlab # [ 243.986698] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 4191884gitlab # [ 243.987412] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.962Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c29b38bc39a29774349c37fb","created_at":"2026-10-25T18:48:25.978Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.980Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c29b38bc39a29774349c37fb: done: 0.025343 sec","job_status":"done","queue_duration_s":1.956601,"scheduling_latency_s":1.956614,"redis_calls":10,"redis_duration_s":0.001188,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000193041,"duration_s":0.025343,"completed_at":"2026-10-25T18:48:27.962Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002928,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1885gitlab # [ 244.028045] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.963Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9d31768f375d1d955f27deda","created_at":"2026-10-25T18:48:25.984Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.987Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9d31768f375d1d955f27deda: start","job_status":"start","queue_duration_s":1.976226,"scheduling_latency_s":1.976239}1886gitlab # [ 244.029530] gitaly[904]: time="2026-10-25T18:48:28.087Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T691RM706FZ3C3M2VFHA77 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.061" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1887gitlab # [ 244.070930] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.983Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9d31768f375d1d955f27deda","created_at":"2026-10-25T18:48:25.984Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.987Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9d31768f375d1d955f27deda: done: 0.02048 sec","job_status":"done","queue_duration_s":1.976226,"scheduling_latency_s":1.976239,"redis_calls":10,"redis_duration_s":0.0012330000000000002,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000192762,"duration_s":0.02048,"completed_at":"2026-10-25T18:48:27.983Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005546,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1888gitlab # [ 244.116153] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.984Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0b9e6252fc2e0e47d6030af3","created_at":"2026-10-25T18:48:25.987Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.990Z","job_size_bytes":6,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0b9e6252fc2e0e47d6030af3: start","job_status":"start","queue_duration_s":1.99471,"scheduling_latency_s":1.994723}1889gitlab # [ 244.116885] gitaly[904]: time="2026-10-25T18:48:28.087Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T691RM706FZ3C3M2VFHA77 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.061" grpc.time_ms=25.741 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1890gitlab # [ 244.117500] nixbot[2636]: {"ts": "2026-10-25T18:48:28.243Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1891gitlab # [ 244.117932] nixbot[2636]: {"ts": "2026-10-25T18:48:28.245Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "7c9d7561d2f49577267b5e147c6e6f8726013972"}1892gitlab # [ 244.118596] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.997Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"0b9e6252fc2e0e47d6030af3","created_at":"2026-10-25T18:48:25.987Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.990Z","job_size_bytes":6,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-0b9e6252fc2e0e47d6030af3: done: 0.012655 sec","job_status":"done","queue_duration_s":1.99471,"scheduling_latency_s":1.994723,"redis_calls":10,"redis_duration_s":0.001192,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199746,"duration_s":0.012655,"completed_at":"2026-10-25T18:48:27.997Z","load_balancing_strategy":"primary","db_duration_s":0.001216,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1893gitlab # [ 244.152924] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:27.998Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0e3a493745d64a836f99e6a5","created_at":"2026-10-25T18:48:25.990Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.993Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0e3a493745d64a836f99e6a5: start","job_status":"start","queue_duration_s":2.0053,"scheduling_latency_s":2.005313}1894gitlab # [ 244.166903] gitlab-workhorse[829]: time="2026-10-25T18:48:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371895gitlab # [ 244.168606] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 1281896gitlab # [ 244.170481] gitaly[904]: time="2026-10-25T18:48:28.127Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH diskcache=77fc5646-6164-4f12-85f3-7c0d531c1a20 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-26T00:48:28.115" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:28.115" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897gitlab # [ 244.179160] gitaly[904]: time="2026-10-25T18:48:28.127Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-26T00:48:28.115" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:28.115" grpc.time_ms=11.389 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898gitlab # [ 244.217295] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.021Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0e3a493745d64a836f99e6a5","created_at":"2026-10-25T18:48:25.990Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.993Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0e3a493745d64a836f99e6a5: done: 0.022932 sec","job_status":"done","queue_duration_s":2.0053,"scheduling_latency_s":2.005313,"redis_calls":10,"redis_duration_s":0.001765,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000237,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000267911,"duration_s":0.022932,"completed_at":"2026-10-25T18:48:28.021Z","load_balancing_strategy":"primary","db_duration_s":0.006593,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1899gitlab # [ 244.256637] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.022Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"3aaebe335245f5d4bf0a8509","created_at":"2026-10-25T18:48:25.997Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.999Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-3aaebe335245f5d4bf0a8509: start","job_status":"start","queue_duration_s":2.022563,"scheduling_latency_s":2.022575}1900gitlab # [ 244.257309] gitaly[904]: time="2026-10-25T18:48:28.150Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:58.149" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:28.149" grpc.time_ms=0.439 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901gitlab # [ 244.258066] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.069Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"3aaebe335245f5d4bf0a8509","created_at":"2026-10-25T18:48:25.997Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:25.999Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-3aaebe335245f5d4bf0a8509: done: 0.046964 sec","job_status":"done","queue_duration_s":2.022563,"scheduling_latency_s":2.022575,"redis_calls":14,"redis_duration_s":0.0034779999999999998,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001625,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001413,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000271542,"duration_s":0.046964,"completed_at":"2026-10-25T18:48:28.069Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1902gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2'1903gitlab # [ 244.311746] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.084Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f9f397c9fddd38ddb5debf95","created_at":"2026-10-25T18:48:26.059Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.061Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f9f397c9fddd38ddb5debf95: start","job_status":"start","queue_duration_s":2.023554,"scheduling_latency_s":2.023573}1904gitlab # [ 244.312658] gitaly[904]: time="2026-10-25T18:48:28.157Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-25T18:48:38.061" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:28.152" grpc.time_ms=5.825 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2', in 0.04 seconds)1906gitlab # [ 244.318072] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.104Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f9f397c9fddd38ddb5debf95","created_at":"2026-10-25T18:48:26.059Z","trace_propagation_headers":{"sentry-trace":"8d094ac190294f5d81d4903dd0a24589-792f99db1fa84a0f","baggage":"sentry-trace_id=8d094ac190294f5d81d4903dd0a24589,sentry-release=Unknown"},"correlation_id":"01M5T67R2KYHXZSR4G1MCHJCN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.061Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f9f397c9fddd38ddb5debf95: done: 0.019525 sec","job_status":"done","queue_duration_s":2.023554,"scheduling_latency_s":2.023573,"redis_calls":8,"redis_duration_s":0.001343,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000344,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019525,"completed_at":"2026-10-25T18:48:28.104Z","load_balancing_strategy":"primary","db_duration_s":0.002689,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1907gitlab # [ 244.373640] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.105Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"de29b7675ecd8ff424324835","created_at":"2026-10-25T18:48:26.126Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-f1f339439b7044dc","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"483fe494dc3e178c913f098d","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.127Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-de29b7675ecd8ff424324835: start","job_status":"start","queue_duration_s":1.977553,"scheduling_latency_s":1.977569}1908gitlab # [ 244.374458] gitaly[904]: time="2026-10-25T18:48:28.179Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-25T18:48:58.178" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:28.178" grpc.time_ms=0.673 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909gitlab # [ 244.380900] nixbot[2636]: {"ts": "2026-10-25T18:48:28.506Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40218 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1\" 200"}1910gitlab # [ 244.382979] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.136Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"de29b7675ecd8ff424324835","created_at":"2026-10-25T18:48:26.126Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-f1f339439b7044dc","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"483fe494dc3e178c913f098d","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.127Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-de29b7675ecd8ff424324835: done: 0.03069 sec","job_status":"done","queue_duration_s":1.977553,"scheduling_latency_s":1.977569,"gitaly_calls":1,"gitaly_duration_s":0.012904,"redis_calls":12,"redis_duration_s":0.0015609999999999999,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000366,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000583,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0203646,"duration_s":0.03069,"completed_at":"2026-10-25T18:48:28.136Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1911gitlab # [ 244.424328] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.137Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e1e4757812b084d656b7b150","created_at":"2026-10-25T18:48:26.220Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-f1f339439b7044dc","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7f000a84311c4cf2f16cf4a1bb8e1b71f11b5816a9bbcffc4e2d3ca60bcf27b4","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.222Z","job_size_bytes":467,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e1e4757812b084d656b7b150: start","job_status":"start","queue_duration_s":1.915061,"scheduling_latency_s":1.915075}1912gitlab # [ 244.472728] gitaly[904]: time="2026-10-25T18:48:28.183Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=38 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-25T18:48:58.181" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:28.182" grpc.time_ms=1.645 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913gitlab # [ 244.473396] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7c9d7561d2f49577267b5e147c6e6f8726013972 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 3161914gitlab # [ 244.473879] gitaly[904]: time="2026-10-25T18:48:28.210Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:38.112" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:28.203" grpc.time_ms=6.899 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab # [ 244.474510] gitaly[904]: time="2026-10-25T18:48:28.217Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T6925STC6N3AVEF2XFR0JG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.201" pid=904 remote_ip="2001:db8:1::1" request_sha=b185ac028c8834b32eccfe0213e03b8f4fd6075c response_bytes=337 span.kind=server system=grpc1916gitlab # [ 244.476436] gitaly[904]: time="2026-10-25T18:48:28.217Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M5T6925STC6N3AVEF2XFR0JG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.201" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1917gitlab # [ 244.488538] gitaly[904]: time="2026-10-25T18:48:28.217Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M5T6925STC6N3AVEF2XFR0JG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.201" grpc.time_ms=15.382 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1918gitlab # [ 244.532614] gitaly[904]: time="2026-10-25T18:48:28.291Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:58.290" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:28.290" grpc.time_ms=0.566 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 244.533258] gitaly[904]: time="2026-10-25T18:48:28.534Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:38.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:28.527" grpc.time_ms=6.758 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab # [ 244.533830] gitaly[904]: time="2026-10-25T18:48:28.626Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M5T692N1X86N527MVY4VZXNS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:58.019" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:28.620" grpc.time_ms=6.016 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921gitlab # [ 244.580913] gitaly[904]: time="2026-10-25T18:48:28.716Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T692N1X86N527MVY4VZXNS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:38.011" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:28.711" grpc.time_ms=5.304 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1922gitlab # [ 244.593683] gitaly[904]: time="2026-10-25T18:48:28.729Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=76 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=76 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=965 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=10 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-25T18:48:58.560" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:28.560" grpc.time_ms=168.514 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923gitlab # [ 244.603474] gitaly[904]: time="2026-10-25T18:48:28.739Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T68WK4HT18GVJXPSFFRETH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-25T18:48:58.731" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:28.732" grpc.time_ms=7.104 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924gitlab # [ 244.612487] sidekiq-cluster[2508]: {"severity":"WARN","time":"2026-10-25T18:48:28.747Z","class":"CreatePipelineWorker","correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"7c9d7561d2f49577267b5e147c6e6f8726013972","after":"a786a2e66da786487d047f8106fd2d599a2fa9a2","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"a786a2e66da786487d047f8106fd2d599a2fa9a2","ref":"refs/heads/master"},"retry":0}1925gitlab # [ 244.615873] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.751Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e1e4757812b084d656b7b150","created_at":"2026-10-25T18:48:26.220Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-f1f339439b7044dc","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7f000a84311c4cf2f16cf4a1bb8e1b71f11b5816a9bbcffc4e2d3ca60bcf27b4","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.222Z","job_size_bytes":467,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e1e4757812b084d656b7b150: done: 0.614124 sec","job_status":"done","queue_duration_s":1.915061,"scheduling_latency_s":1.915075,"gitaly_calls":7,"gitaly_duration_s":0.204039,"redis_calls":33,"redis_duration_s":0.00315,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000389,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001425,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000626,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.390208,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.614124,"completed_at":"2026-10-25T18:48:28.751Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014102,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1926gitlab # [ 244.627678] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.752Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"1e2b5ba3-c9a8-4b36-839a-a5a1af5f9456\"}"],"class":"WebHookWorker","jid":"a1711510b1b631148ceb0468","created_at":"2026-10-25T18:48:26.233Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-f1f339439b7044dc","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3bc3c55dd8f5539805d1c08d61a04a61e1638f14c255dd85058d92e4269a577","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.234Z","job_size_bytes":1749,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a1711510b1b631148ceb0468: start","job_status":"start","queue_duration_s":2.518109,"scheduling_latency_s":2.518127}1927gitlab # [ 244.674708] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.777Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"1e2b5ba3-c9a8-4b36-839a-a5a1af5f9456\"}"],"class":"WebHookWorker","jid":"a1711510b1b631148ceb0468","created_at":"2026-10-25T18:48:26.233Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-f1f339439b7044dc","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3bc3c55dd8f5539805d1c08d61a04a61e1638f14c255dd85058d92e4269a577","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.234Z","job_size_bytes":1749,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a1711510b1b631148ceb0468: done: 0.025404 sec","job_status":"done","queue_duration_s":2.518109,"scheduling_latency_s":2.518127,"redis_calls":14,"redis_duration_s":0.001712,"redis_read_bytes":25,"redis_write_bytes":4681,"redis_queues_calls":2,"redis_queues_duration_s":0.00024,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3725,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.00525876899999389,"cpu_s":0.015325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.025404,"completed_at":"2026-10-25T18:48:28.777Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1928gitlab # [ 244.687140] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.779Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"11909be993059e5216b6d4eb","created_at":"2026-10-25T18:48:26.383Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-032d3cc3b2a94ec9","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.386Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-11909be993059e5216b6d4eb: start","job_status":"start","queue_duration_s":2.392466,"scheduling_latency_s":2.392488}1929gitlab # [ 244.699534] nixbot[2636]: {"ts": "2026-10-25T18:48:28.768Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "ef898b2a-82f8-4625-a689-eb8347712087", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "a786a2e66da786487d047f8106fd2d599a2fa9a2", "pr": null}1930gitlab # [ 244.705576] nixbot[2636]: {"ts": "2026-10-25T18:48:28.770Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40222 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1931gitlab # [ 244.706216] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.809Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"11909be993059e5216b6d4eb","created_at":"2026-10-25T18:48:26.383Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-032d3cc3b2a94ec9","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:26.386Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-11909be993059e5216b6d4eb: done: 0.030503 sec","job_status":"done","queue_duration_s":2.392466,"scheduling_latency_s":2.392488,"redis_calls":19,"redis_duration_s":0.002617,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000709,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000311,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001151,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004670705,"duration_s":0.030503,"completed_at":"2026-10-25T18:48:28.809Z","load_balancing_strategy":"primary","db_duration_s":0.004853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1932gitlab # [ 244.779111] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.814Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d7d84477dcba74050127e7f3"],"class":"WebHooks::LogExecutionWorker","jid":"71abb2cc36298b47a6420a90","created_at":"2026-10-25T18:48:27.550Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-3433bb4929b74f4c","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1be69aaddd75b84c3ddf6b0baf09ad103c5273cc3ee631327f96676fe26c8c80","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:27.556Z","job_size_bytes":2592,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-71abb2cc36298b47a6420a90: start","job_status":"start","queue_duration_s":1.258003,"scheduling_latency_s":1.258017}1933gitlab # [ 244.779830] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.902Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d7d84477dcba74050127e7f3"],"class":"WebHooks::LogExecutionWorker","jid":"71abb2cc36298b47a6420a90","created_at":"2026-10-25T18:48:27.550Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-3433bb4929b74f4c","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1be69aaddd75b84c3ddf6b0baf09ad103c5273cc3ee631327f96676fe26c8c80","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:27.556Z","job_size_bytes":2592,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-71abb2cc36298b47a6420a90: done: 0.088438 sec","job_status":"done","queue_duration_s":1.258003,"scheduling_latency_s":1.258017,"redis_calls":10,"redis_duration_s":0.004863,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000469,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001827,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002567,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025534,"duration_s":0.088438,"completed_at":"2026-10-25T18:48:28.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.096643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1934gitlab # [ 244.801051] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:28.903Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"39b0966e9b7596a7d56455cb","created_at":"2026-10-25T18:48:28.557Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-71aecd93ca6e4873","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:28.560Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-39b0966e9b7596a7d56455cb: start","job_status":"start","queue_duration_s":0.343723,"scheduling_latency_s":0.343738}1935gitlab # [ 244.813545] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 601936gitlab # [ 244.815300] gitlab-workhorse[829]: time="2026-10-25T18:48:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671937gitlab # [ 244.816778] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 441938gitlab # [ 244.818549] gitaly[904]: time="2026-10-25T18:48:28.846Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M5T692VJCMSEFZ6F5AJZ4Q4Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.840" grpc.time_ms=6.136 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1939gitlab # [ 244.859549] gitaly[904]: time="2026-10-25T18:48:28.938Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T692YY8HSBF0JC0B8J882Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.932" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1940gitlab # [ 244.860137] gitaly[904]: time="2026-10-25T18:48:28.938Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T692YY8HSBF0JC0B8J882Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.932" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1941gitlab # [ 244.860794] gitaly[904]: time="2026-10-25T18:48:28.938Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T692YY8HSBF0JC0B8J882Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:28.932" grpc.time_ms=5.989 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1942gitlab # [ 244.931438] gitaly[904]: time="2026-10-25T18:48:29.067Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:29.066" grpc.time_ms=0.318 pid=904 span.kind=server system=grpc1943gitlab # [ 244.945115] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7c9d7561d2f49577267b5e147c6e6f8726013972 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 5021944gitlab # [ 244.961177] gitaly[904]: time="2026-10-25T18:48:29.095Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:29.094" grpc.time_ms=0.287 pid=904 span.kind=server system=grpc1945gitlab # [ 244.971989] gitaly[904]: time="2026-10-25T18:48:29.107Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T6931V1BRNYHXW9PT4SC28 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.100" grpc.time_ms=7.21 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1946gitlab # [ 244.972755] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1201947gitlab # [ 244.981706] gitaly[904]: time="2026-10-25T18:48:29.117Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:29.117" grpc.time_ms=0.289 pid=904 span.kind=server system=grpc1948gitlab # [ 244.991481] gitaly[904]: time="2026-10-25T18:48:29.126Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR diskcache=dec8f254-2624-4861-8e41-d9c2e8a82d08 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-25T18:48:59.046" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:29.047" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1949gitlab # [ 244.992165] gitaly[904]: time="2026-10-25T18:48:29.127Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=52 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6874 command.oublock=0 command.real_time_ms=72 command.spawn_token_fork_ms=0 command.system_time_ms=31 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-25T18:48:59.046" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:29.047" grpc.time_ms=79.695 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1950gitlab # [ 245.022063] gitaly[904]: time="2026-10-25T18:48:29.157Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T6935NGQEN6B35CTJA8RZH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.149" pid=904 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1951gitlab # [ 245.023164] gitaly[904]: time="2026-10-25T18:48:29.157Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T6935NGQEN6B35CTJA8RZH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.149" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1952gitlab # [ 245.024143] gitaly[904]: time="2026-10-25T18:48:29.158Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T6935NGQEN6B35CTJA8RZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.149" grpc.time_ms=9.036 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1953gitlab # [ 245.024824] gitlab-workhorse[829]: time="2026-10-25T18:48:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671954gitlab # [ 245.025494] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 491955gitlab # [ 245.038333] gitaly[904]: time="2026-10-25T18:48:29.174Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-25T18:48:59.173" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:29.173" grpc.time_ms=0.502 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1956gitlab # [ 245.059475] gitaly[904]: time="2026-10-25T18:48:29.195Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:59.193" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:29.194" grpc.time_ms=0.522 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957gitlab # [ 245.095209] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.229Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a7d561b48a0e30a51007fcf5","created_at":"2026-10-25T18:48:29.227Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"39b0966e9b7596a7d56455cb","job_size_bytes":3,"pid":2508,"job_status":"deduplicated","message":"PipelineProcessWorker JID-a7d561b48a0e30a51007fcf5: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1958gitlab # [ 245.098064] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.233Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"39b0966e9b7596a7d56455cb","created_at":"2026-10-25T18:48:28.557Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-71aecd93ca6e4873","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:28.560Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2508,"job_status":"rescheduled","message":"PipelineProcessWorker JID-39b0966e9b7596a7d56455cb: rescheduled"}1959gitlab # [ 245.103978] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.239Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"39b0966e9b7596a7d56455cb","created_at":"2026-10-25T18:48:28.557Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-71aecd93ca6e4873","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:28.560Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-39b0966e9b7596a7d56455cb: done: 0.335616 sec","job_status":"done","queue_duration_s":0.343723,"scheduling_latency_s":0.343738,"gitaly_calls":3,"gitaly_duration_s":0.088539,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.017195,"redis_read_bytes":228,"redis_write_bytes":18463,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001081,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001067,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00443,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12309,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005982,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001481,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000602,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001471,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.087,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":18,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.034,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.034,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.024,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.105724,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.269047323,"duration_s":0.335616,"completed_at":"2026-10-25T18:48:29.239Z","load_balancing_strategy":"primary","db_duration_s":0.113928,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1960gitlab # [ 245.158950] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.241Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a1711510b1b631148ceb0468"],"class":"WebHooks::LogExecutionWorker","jid":"83223c452868b91dfe171178","created_at":"2026-10-25T18:48:28.771Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-f7396659c5774a32","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cd5455023ad744cd894033108993c5ef06d2f933efa77c8ea516e51b7b2f519b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:28.772Z","job_size_bytes":2592,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-83223c452868b91dfe171178: start","job_status":"start","queue_duration_s":0.468017,"scheduling_latency_s":0.468035}1961gitlab # [ 245.159566] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.260Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a1711510b1b631148ceb0468"],"class":"WebHooks::LogExecutionWorker","jid":"83223c452868b91dfe171178","created_at":"2026-10-25T18:48:28.771Z","trace_propagation_headers":{"sentry-trace":"00c676d586da41478272032ee5e3fa73-f7396659c5774a32","baggage":"sentry-trace_id=00c676d586da41478272032ee5e3fa73,sentry-release=Unknown"},"correlation_id":"01M5T68WK4HT18GVJXPSFFRETH","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cd5455023ad744cd894033108993c5ef06d2f933efa77c8ea516e51b7b2f519b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:28.772Z","job_size_bytes":2592,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-83223c452868b91dfe171178: done: 0.019086 sec","job_status":"done","queue_duration_s":0.468017,"scheduling_latency_s":0.468035,"redis_calls":7,"redis_duration_s":0.002177,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019086,"completed_at":"2026-10-25T18:48:29.260Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002723,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 245.214872] gitaly[904]: time="2026-10-25T18:48:29.243Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T6934ZBXYPYQS8GGSDNAYG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:59.241" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:29.241" grpc.time_ms=1.078 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 245.215661] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7c9d7561d2f49577267b5e147c6e6f8726013972 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2271964gitlab # [ 245.216104] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1361965gitlab # [ 245.216618] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.261Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"e6becffd260c64a583843c5e","created_at":"2026-10-25T18:48:28.801Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-005f76c3cfc14967","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:28.804Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-e6becffd260c64a583843c5e: start","job_status":"start","queue_duration_s":0.456494,"scheduling_latency_s":0.456509}1966gitlab # [ 245.217624] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.331Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"e6becffd260c64a583843c5e","created_at":"2026-10-25T18:48:28.801Z","trace_propagation_headers":{"sentry-trace":"2ffa8534260c4d4cb2f46e68ce55ba3a-005f76c3cfc14967","baggage":"sentry-trace_id=2ffa8534260c4d4cb2f46e68ce55ba3a,sentry-release=Unknown"},"correlation_id":"01M5T67KN64FY0XE0NQ3YJF8NS","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:28.804Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-e6becffd260c64a583843c5e: done: 0.070183 sec","job_status":"done","queue_duration_s":0.456494,"scheduling_latency_s":0.456509,"redis_calls":11,"redis_duration_s":0.001999,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000361,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000220419,"duration_s":0.070183,"completed_at":"2026-10-25T18:48:29.331Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1967gitlab # [ 245.262082] gitaly[904]: time="2026-10-25T18:48:29.272Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M5T6934ZBXYPYQS8GGSDNAYG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:39.156" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:29.266" grpc.time_ms=5.927 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 245.263494] gitlab-workhorse[829]: time="2026-10-25T18:48:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671969gitlab # [ 245.265155] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 501970gitlab # [ 245.266854] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.332Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f4d0bf62eda3566c0785c72a","created_at":"2026-10-25T18:48:29.041Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.043Z","job_size_bytes":23,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f4d0bf62eda3566c0785c72a: start","job_status":"start","queue_duration_s":0.289633,"scheduling_latency_s":0.289659}1971gitlab # [ 245.281840] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.364Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f4d0bf62eda3566c0785c72a","created_at":"2026-10-25T18:48:29.041Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.043Z","job_size_bytes":23,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f4d0bf62eda3566c0785c72a: done: 0.032144 sec","job_status":"done","queue_duration_s":0.289633,"scheduling_latency_s":0.289659,"redis_calls":7,"redis_duration_s":0.0011560000000000001,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032144,"completed_at":"2026-10-25T18:48:29.364Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008722,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1972gitlab # [ 245.345839] gitaly[904]: time="2026-10-25T18:48:29.318Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=79 command.spawn_token_fork_ms=75 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M5T6937YRPH609Z12GWR5NDP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.237" grpc.time_ms=80.63 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1973gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2'1974gitlab # [ 245.346606] gitaly[904]: time="2026-10-25T18:48:29.369Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T693C8EBRD0XFRBG2XRRSE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.363" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1975gitlab # [ 245.347097] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.366Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d4d4eb4b088b5a4b0d21d307","created_at":"2026-10-25T18:48:29.132Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.134Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d4d4eb4b088b5a4b0d21d307: start","job_status":"start","queue_duration_s":0.231361,"scheduling_latency_s":0.231382}1976gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2', in 0.04 seconds)1977gitlab # [ 245.355649] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.385Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d4d4eb4b088b5a4b0d21d307","created_at":"2026-10-25T18:48:29.132Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.134Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d4d4eb4b088b5a4b0d21d307: done: 0.019553 sec","job_status":"done","queue_duration_s":0.231361,"scheduling_latency_s":0.231382,"redis_calls":7,"redis_duration_s":0.0013830000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000331,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019553,"completed_at":"2026-10-25T18:48:29.385Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978gitlab # [ 245.418208] gitaly[904]: time="2026-10-25T18:48:29.369Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693C8EBRD0XFRBG2XRRSE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.363" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1979gitlab # [ 245.419209] nixbot[2636]: {"ts": "2026-10-25T18:48:29.549Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40228 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1\" 200"}1980gitlab # [ 245.419778] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.386Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e8081058f48c79b56434361c","created_at":"2026-10-25T18:48:29.141Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.144Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e8081058f48c79b56434361c: start","job_status":"start","queue_duration_s":0.24184,"scheduling_latency_s":0.241855}1981gitlab # [ 245.421234] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.408Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e8081058f48c79b56434361c","created_at":"2026-10-25T18:48:29.141Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.144Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e8081058f48c79b56434361c: done: 0.022277 sec","job_status":"done","queue_duration_s":0.24184,"scheduling_latency_s":0.241855,"redis_calls":7,"redis_duration_s":0.002235,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001494,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000741,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022277,"completed_at":"2026-10-25T18:48:29.408Z","load_balancing_strategy":"primary","db_duration_s":0.00739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1982gitlab # [ 245.464182] gitaly[904]: time="2026-10-25T18:48:29.370Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693C8EBRD0XFRBG2XRRSE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.363" grpc.time_ms=6.749 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1983gitlab # [ 245.504772] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 3801984gitlab # [ 245.505690] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1761985gitlab # [ 245.506192] gitlab-workhorse[829]: time="2026-10-25T18:48:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161986gitlab # [ 245.506747] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 501987gitlab # [ 245.507450] gitlab-workhorse[829]: time="2026-10-25T18:48:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2211988gitlab # [ 245.508038] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 221 "" "git/2.55.0" 571989gitlab # [ 245.508643] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.412Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"da81d333faf3be2045e8b4be","created_at":"2026-10-25T18:48:29.150Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.153Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-da81d333faf3be2045e8b4be: start","job_status":"start","queue_duration_s":0.258941,"scheduling_latency_s":0.258957}1990gitlab # [ 245.509663] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.434Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"da81d333faf3be2045e8b4be","created_at":"2026-10-25T18:48:29.150Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.153Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-da81d333faf3be2045e8b4be: done: 0.022848 sec","job_status":"done","queue_duration_s":0.258941,"scheduling_latency_s":0.258957,"redis_calls":10,"redis_duration_s":0.002915,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001051,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022848,"completed_at":"2026-10-25T18:48:29.434Z","load_balancing_strategy":"primary","db_duration_s":0.003907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991gitlab # [ 245.551343] gitaly[904]: time="2026-10-25T18:48:29.568Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=133 command.spawn_token_fork_ms=131 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T69385W1J93HGAE6M0WKC7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.433" grpc.time_ms=134.788 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1992gitlab # [ 245.552265] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.436Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e978d0a236515cd1c5e1fa0a","created_at":"2026-10-25T18:48:29.153Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.155Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e978d0a236515cd1c5e1fa0a: start","job_status":"start","queue_duration_s":0.28105,"scheduling_latency_s":0.281079}1993gitlab # [ 245.562732] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.450Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e978d0a236515cd1c5e1fa0a","created_at":"2026-10-25T18:48:29.153Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.155Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e978d0a236515cd1c5e1fa0a: done: 0.01464 sec","job_status":"done","queue_duration_s":0.28105,"scheduling_latency_s":0.281079,"redis_calls":7,"redis_duration_s":0.000895,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01464,"completed_at":"2026-10-25T18:48:29.450Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1994gitlab # [ 245.621707] gitaly[904]: time="2026-10-25T18:48:29.569Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=145 command.spawn_token_fork_ms=141 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T693EH1BACGVG3FMMNN06X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.423" grpc.time_ms=145.886 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1995gitlab # [ 245.622432] gitaly[904]: time="2026-10-25T18:48:29.579Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:29.579" grpc.time_ms=0.285 pid=904 span.kind=server system=grpc1996gitlab # [ 245.623399] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.451Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1c4079370ea5b1791c6d8742","created_at":"2026-10-25T18:48:29.156Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.157Z","job_size_bytes":19,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1c4079370ea5b1791c6d8742: start","job_status":"start","queue_duration_s":0.294254,"scheduling_latency_s":0.294268}1997gitlab # [ 245.636085] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.466Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1c4079370ea5b1791c6d8742","created_at":"2026-10-25T18:48:29.156Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.157Z","job_size_bytes":19,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1c4079370ea5b1791c6d8742: done: 0.015038 sec","job_status":"done","queue_duration_s":0.294254,"scheduling_latency_s":0.294268,"redis_calls":8,"redis_duration_s":0.000973,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015038,"completed_at":"2026-10-25T18:48:29.466Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1998gitlab # [ 245.703741] gitaly[904]: time="2026-10-25T18:48:29.622Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T693M3WK20EAA3KH729GPN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.607" pid=904 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc1999gitlab # [ 245.705275] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.467Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"019e517fc3cd65baad573ab8","created_at":"2026-10-25T18:48:29.158Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.159Z","job_size_bytes":66,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-019e517fc3cd65baad573ab8: start","job_status":"start","queue_duration_s":0.308305,"scheduling_latency_s":0.308319}2000gitlab # [ 245.705929] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.480Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"019e517fc3cd65baad573ab8","created_at":"2026-10-25T18:48:29.158Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.159Z","job_size_bytes":66,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-019e517fc3cd65baad573ab8: done: 0.012902 sec","job_status":"done","queue_duration_s":0.308305,"scheduling_latency_s":0.308319,"redis_calls":7,"redis_duration_s":0.000875,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012902,"completed_at":"2026-10-25T18:48:29.480Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2001gitlab # [ 245.748219] gitaly[904]: time="2026-10-25T18:48:29.622Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693M3WK20EAA3KH729GPN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.607" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc2002gitlab # [ 245.751294] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.481Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"801e6f1c8f09dacbb1a7d360","created_at":"2026-10-25T18:48:29.160Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.162Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-801e6f1c8f09dacbb1a7d360: start","job_status":"start","queue_duration_s":0.319221,"scheduling_latency_s":0.319235}2003gitlab # [ 245.765630] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.496Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"801e6f1c8f09dacbb1a7d360","created_at":"2026-10-25T18:48:29.160Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.162Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-801e6f1c8f09dacbb1a7d360: done: 0.015181 sec","job_status":"done","queue_duration_s":0.319221,"scheduling_latency_s":0.319235,"redis_calls":7,"redis_duration_s":0.000922,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015181,"completed_at":"2026-10-25T18:48:29.496Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab # [ 245.814784] gitaly[904]: time="2026-10-25T18:48:29.622Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693M3WK20EAA3KH729GPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.607" grpc.time_ms=14.499 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc2005gitlab # [ 245.816461] nixbot[2636]: {"ts": "2026-10-25T18:48:29.910Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-a786a2e66da7-6"}2006gitlab # [ 245.819470] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.497Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"99881d2be6eb523b06e3f295","created_at":"2026-10-25T18:48:29.162Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.163Z","job_size_bytes":6,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-99881d2be6eb523b06e3f295: start","job_status":"start","queue_duration_s":0.333874,"scheduling_latency_s":0.333887}2007gitlab # [ 245.832531] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.505Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"99881d2be6eb523b06e3f295","created_at":"2026-10-25T18:48:29.162Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.163Z","job_size_bytes":6,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-99881d2be6eb523b06e3f295: done: 0.007846 sec","job_status":"done","queue_duration_s":0.333874,"scheduling_latency_s":0.333887,"redis_calls":7,"redis_duration_s":0.000884,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000229,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007846,"completed_at":"2026-10-25T18:48:29.505Z","load_balancing_strategy":"primary","db_duration_s":0.000245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 245.881037] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 2302009gitlab # [ 245.881813] gitaly[904]: time="2026-10-25T18:48:29.628Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M5T693M3WK20EAA3KETS0Z4G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.612" pid=904 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=221 span.kind=server system=grpc2010gitlab # [ 245.882512] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.505Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"02bef12582c1ee28cf192ada","created_at":"2026-10-25T18:48:29.163Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.164Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-02bef12582c1ee28cf192ada: start","job_status":"start","queue_duration_s":0.340962,"scheduling_latency_s":0.340974}2011gitlab # [ 245.896429] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.518Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"02bef12582c1ee28cf192ada","created_at":"2026-10-25T18:48:29.163Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.164Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-02bef12582c1ee28cf192ada: done: 0.01243 sec","job_status":"done","queue_duration_s":0.340962,"scheduling_latency_s":0.340974,"redis_calls":7,"redis_duration_s":0.000858,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000624,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01243,"completed_at":"2026-10-25T18:48:29.518Z","load_balancing_strategy":"primary","db_duration_s":0.001772,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2012gitlab # [ 245.948186] nixbot[2636]: {"ts": "2026-10-25T18:48:29.997Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-a786a2e66da7-6", "argv0": "bwrap"}2013gitlab # [ 245.948831] gitaly[904]: time="2026-10-25T18:48:29.628Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693M3WK20EAA3KETS0Z4G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.612" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc2014gitlab # [ 245.949508] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.519Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7ad85070601a2ba9e5e24cf3","created_at":"2026-10-25T18:48:29.233Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.234Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7ad85070601a2ba9e5e24cf3: start","job_status":"start","queue_duration_s":0.284599,"scheduling_latency_s":0.284612}2015gitlab # [ 245.961118] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.783Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7ad85070601a2ba9e5e24cf3","created_at":"2026-10-25T18:48:29.233Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.234Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2508,"job_status":"rescheduled","message":"PipelineProcessWorker JID-7ad85070601a2ba9e5e24cf3: rescheduled"}2016gitlab # [ 245.974713] gitaly[904]: time="2026-10-25T18:48:29.629Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693M3WK20EAA3KETS0Z4G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-25T18:48:29.612" grpc.time_ms=17.134 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc2017gitlab # [ 246.018884] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.788Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7ad85070601a2ba9e5e24cf3","created_at":"2026-10-25T18:48:29.233Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-3a9cf01d5d254e22","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.234Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7ad85070601a2ba9e5e24cf3: done: 0.269048 sec","job_status":"done","queue_duration_s":0.284599,"scheduling_latency_s":0.284612,"gitaly_calls":3,"gitaly_duration_s":0.153467,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.013497,"redis_read_bytes":146,"redis_write_bytes":18014,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000325,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001128,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002065,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12313,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.006015,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00015,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000456,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003033,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076152,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.247321377,"duration_s":0.269048,"completed_at":"2026-10-25T18:48:29.788Z","load_balancing_strategy":"primary","db_duration_s":0.026937,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2018gitlab # [ 246.033995] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.790Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d53f94a245cb48a43f620b22","created_at":"2026-10-25T18:48:29.553Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.554Z","job_size_bytes":23,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d53f94a245cb48a43f620b22: start","job_status":"start","queue_duration_s":0.236407,"scheduling_latency_s":0.236492}2019gitlab # [ 246.077143] gitaly[904]: time="2026-10-25T18:48:29.631Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:29.630" grpc.time_ms=0.329 pid=904 span.kind=server system=grpc2020gitlab # [ 246.077790] gitaly[904]: time="2026-10-25T18:48:29.689Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:29.688" grpc.time_ms=0.886 pid=904 span.kind=server system=grpc2021gitlab # [ 246.078416] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.820Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d53f94a245cb48a43f620b22","created_at":"2026-10-25T18:48:29.553Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.554Z","job_size_bytes":23,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d53f94a245cb48a43f620b22: done: 0.029305 sec","job_status":"done","queue_duration_s":0.236407,"scheduling_latency_s":0.236492,"redis_calls":7,"redis_duration_s":0.000957,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029305,"completed_at":"2026-10-25T18:48:29.819Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004846,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2022gitlab # [ 246.124060] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.820Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c5bce8e46dbe6397d1dfd13b","created_at":"2026-10-25T18:48:29.711Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.713Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c5bce8e46dbe6397d1dfd13b: start","job_status":"start","queue_duration_s":0.106997,"scheduling_latency_s":0.107014}2023gitlab # [ 246.124734] gitaly[904]: time="2026-10-25T18:48:29.704Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR diskcache=311940ce-06db-45d3-ad47-b743ed10d529 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-25T18:48:59.557" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:29.557" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2024gitlab # [ 246.126919] gitaly[904]: time="2026-10-25T18:48:29.704Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7052 command.oublock=0 command.real_time_ms=133 command.spawn_token_fork_ms=0 command.system_time_ms=41 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-25T18:48:59.557" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:29.557" grpc.time_ms=146.187 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025gitlab # [ 246.138840] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.893Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c5bce8e46dbe6397d1dfd13b","created_at":"2026-10-25T18:48:29.711Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.713Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c5bce8e46dbe6397d1dfd13b: done: 0.072111 sec","job_status":"done","queue_duration_s":0.106997,"scheduling_latency_s":0.107014,"redis_calls":7,"redis_duration_s":0.001047,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.072111,"completed_at":"2026-10-25T18:48:29.893Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026gitlab # [ 246.187979] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.894Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"17242a86ec98b608aa1f1152","created_at":"2026-10-25T18:48:29.720Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.724Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-17242a86ec98b608aa1f1152: start","job_status":"start","queue_duration_s":0.169604,"scheduling_latency_s":0.169621}2027gitlab # [ 246.190615] gitaly[904]: time="2026-10-25T18:48:29.714Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:59.012" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:29.712" grpc.time_ms=1.553 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2028gitlab # [ 246.201866] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.912Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"17242a86ec98b608aa1f1152","created_at":"2026-10-25T18:48:29.720Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.724Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-17242a86ec98b608aa1f1152: done: 0.018315 sec","job_status":"done","queue_duration_s":0.169604,"scheduling_latency_s":0.169621,"redis_calls":7,"redis_duration_s":0.001269,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018315,"completed_at":"2026-10-25T18:48:29.912Z","load_balancing_strategy":"primary","db_duration_s":0.003462,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2029gitlab # [ 246.264210] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.913Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f518c1230ebf7e7356fe0e62","created_at":"2026-10-25T18:48:29.730Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.732Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f518c1230ebf7e7356fe0e62: start","job_status":"start","queue_duration_s":0.180282,"scheduling_latency_s":0.180296}2030gitlab # [ 246.264908] gitaly[904]: time="2026-10-25T18:48:29.730Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-25T18:48:59.021" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:29.722" grpc.time_ms=7.675 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2031gitlab # [ 246.265499] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.931Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f518c1230ebf7e7356fe0e62","created_at":"2026-10-25T18:48:29.730Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.732Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f518c1230ebf7e7356fe0e62: done: 0.018618 sec","job_status":"done","queue_duration_s":0.180282,"scheduling_latency_s":0.180296,"redis_calls":10,"redis_duration_s":0.001232,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000355,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000229,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018618,"completed_at":"2026-10-25T18:48:29.931Z","load_balancing_strategy":"primary","db_duration_s":0.003701,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2032gitlab # [ 246.328985] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.933Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"76e104405f13ba0a54ad3731","created_at":"2026-10-25T18:48:29.733Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.734Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-76e104405f13ba0a54ad3731: start","job_status":"start","queue_duration_s":0.198905,"scheduling_latency_s":0.198918}2033gitlab # [ 246.329772] gitaly[904]: time="2026-10-25T18:48:29.739Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:39.064" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:29.734" grpc.time_ms=5.347 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2034gitlab # [ 246.330961] nix-daemon[2801]: accepted connection from pid 3664, user nixbot2035gitlab # [ 246.331642] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"76e104405f13ba0a54ad3731","created_at":"2026-10-25T18:48:29.733Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.734Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-76e104405f13ba0a54ad3731: done: 0.014493 sec","job_status":"done","queue_duration_s":0.198905,"scheduling_latency_s":0.198918,"redis_calls":7,"redis_duration_s":0.000934,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000233,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014493,"completed_at":"2026-10-25T18:48:29.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002635,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2036gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2'2037gitlab # [ 246.391447] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.949Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"2d60e22289ee38c793fc05c2","created_at":"2026-10-25T18:48:29.734Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.736Z","job_size_bytes":19,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-2d60e22289ee38c793fc05c2: start","job_status":"start","queue_duration_s":0.212898,"scheduling_latency_s":0.212912}2038gitlab # [ 246.392124] gitaly[904]: time="2026-10-25T18:48:29.751Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-25T18:48:59.750" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:29.750" grpc.time_ms=0.624 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2039gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2', in 0.04 seconds)2040(finished: subtest: push delivers the webhook, in 46.81 seconds)2041subtest: eval and build post commit statuses2042gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses2043gitlab # [ 246.395263] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.971Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"2d60e22289ee38c793fc05c2","created_at":"2026-10-25T18:48:29.734Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.736Z","job_size_bytes":19,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-2d60e22289ee38c793fc05c2: done: 0.022654 sec","job_status":"done","queue_duration_s":0.212898,"scheduling_latency_s":0.212912,"redis_calls":8,"redis_duration_s":0.004043,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003757,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022654,"completed_at":"2026-10-25T18:48:29.971Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003876,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2044gitlab # [ 246.432971] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.975Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7cb2224f395727ad7b981f6c","created_at":"2026-10-25T18:48:29.736Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.737Z","job_size_bytes":66,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7cb2224f395727ad7b981f6c: start","job_status":"start","queue_duration_s":0.23737,"scheduling_latency_s":0.23739}2045gitlab # [ 246.480748] gitaly[904]: time="2026-10-25T18:48:29.752Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-25T18:48:39.082" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:29.752" grpc.time_ms=0.165 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2046gitlab # [ 246.481883] nixbot[2636]: {"ts": "2026-10-25T18:48:30.592Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40236 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1\" 200"}2047gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses, in 0.07 seconds)2048gitlab # [ 246.482597] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.990Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7cb2224f395727ad7b981f6c","created_at":"2026-10-25T18:48:29.736Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.737Z","job_size_bytes":66,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7cb2224f395727ad7b981f6c: done: 0.015561 sec","job_status":"done","queue_duration_s":0.23737,"scheduling_latency_s":0.23739,"redis_calls":7,"redis_duration_s":0.0009580000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015561,"completed_at":"2026-10-25T18:48:29.990Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049gitlab # [ 246.497683] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:29.991Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d841e18280197a780876a4a8","created_at":"2026-10-25T18:48:29.738Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.739Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d841e18280197a780876a4a8: start","job_status":"start","queue_duration_s":0.252436,"scheduling_latency_s":0.25245}2050gitlab # [ 246.512299] gitaly[904]: time="2026-10-25T18:48:29.762Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-25T18:48:39.085" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:29.755" grpc.time_ms=6.601 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2051gitlab # [ 246.524520] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.007Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d841e18280197a780876a4a8","created_at":"2026-10-25T18:48:29.738Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.739Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d841e18280197a780876a4a8: done: 0.016545 sec","job_status":"done","queue_duration_s":0.252436,"scheduling_latency_s":0.25245,"redis_calls":7,"redis_duration_s":0.0009500000000000001,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016545,"completed_at":"2026-10-25T18:48:30.007Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003928,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2052gitlab # [ 246.588621] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.008Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3b54b26f47a37a04e479bc2a","created_at":"2026-10-25T18:48:29.739Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.740Z","job_size_bytes":6,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3b54b26f47a37a04e479bc2a: start","job_status":"start","queue_duration_s":0.268554,"scheduling_latency_s":0.268567}2053gitlab # [ 246.589436] gitaly[904]: time="2026-10-25T18:48:29.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T692AYZHY6QTZKP06Q7YVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:59.776" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:29.776" grpc.time_ms=0.472 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2054gitlab # [ 246.590038] gitaly[904]: time="2026-10-25T18:48:29.804Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:29.804" grpc.time_ms=0.321 pid=904 span.kind=server system=grpc2055gitlab # [ 246.595890] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 512056gitlab # [ 246.636963] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.020Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3b54b26f47a37a04e479bc2a","created_at":"2026-10-25T18:48:29.739Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.740Z","job_size_bytes":6,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3b54b26f47a37a04e479bc2a: done: 0.011481 sec","job_status":"done","queue_duration_s":0.268554,"scheduling_latency_s":0.268567,"redis_calls":7,"redis_duration_s":0.003833,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002824,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011481,"completed_at":"2026-10-25T18:48:30.020Z","load_balancing_strategy":"primary","db_duration_s":0.00032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057gitlab # [ 246.637515] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.024Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f674bf433d178c70ce1c24a5","created_at":"2026-10-25T18:48:29.740Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.741Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f674bf433d178c70ce1c24a5: start","job_status":"start","queue_duration_s":0.282718,"scheduling_latency_s":0.282733}2058gitlab # [ 246.675783] gitaly[904]: time="2026-10-25T18:48:29.827Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-25T18:48:29.826" grpc.time_ms=0.396 pid=904 span.kind=server system=grpc2059gitlab # [ 246.676607] gitaly[904]: time="2026-10-25T18:48:29.829Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:48:39.093" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:29.764" operations="[repack_refs]" pid=904 ref=refs/keep-around/a786a2e66da786487d047f8106fd2d599a2fa9a2 remote_ip="2001:db8:1::1" revision=a786a2e66da786487d047f8106fd2d599a2fa9a2 span.kind=server system=grpc username=root2060gitlab # [ 246.677252] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.045Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f674bf433d178c70ce1c24a5","created_at":"2026-10-25T18:48:29.740Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.741Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f674bf433d178c70ce1c24a5: done: 0.02117 sec","job_status":"done","queue_duration_s":0.282718,"scheduling_latency_s":0.282733,"redis_calls":7,"redis_duration_s":0.004285,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003074,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001211,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02117,"completed_at":"2026-10-25T18:48:30.045Z","load_balancing_strategy":"primary","db_duration_s":0.004948,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2061gitlab # [ 246.724665] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.046Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f54b7df22861ade156f57c15","created_at":"2026-10-25T18:48:29.783Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.784Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f54b7df22861ade156f57c15: start","job_status":"start","queue_duration_s":0.262103,"scheduling_latency_s":0.262117}2062gitlab # [ 246.725535] gitaly[904]: time="2026-10-25T18:48:29.831Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:48:39.093" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:29.764" pid=904 ref=refs/keep-around/a786a2e66da786487d047f8106fd2d599a2fa9a2 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-10-25 18:47:33.26703294 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" revision=a786a2e66da786487d047f8106fd2d599a2fa9a2 span.kind=server system=grpc username=root2063gitlab # [ 246.730561] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.061Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f54b7df22861ade156f57c15","created_at":"2026-10-25T18:48:29.783Z","trace_propagation_headers":{"sentry-trace":"382a5063d8c94a16b1d28ea27bd94723-b9daac3e21694c5f","baggage":"sentry-trace_id=382a5063d8c94a16b1d28ea27bd94723,sentry-release=Unknown"},"correlation_id":"01M5T692AYZHY6QTZKP06Q7YVR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.784Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f54b7df22861ade156f57c15: done: 0.01551 sec","job_status":"done","queue_duration_s":0.262103,"scheduling_latency_s":0.262117,"redis_calls":8,"redis_duration_s":0.001048,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01551,"completed_at":"2026-10-25T18:48:30.061Z","load_balancing_strategy":"primary","db_duration_s":0.002116,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2064gitlab # [ 246.768736] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.062Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"07a3642ae51f83f74c1bfddd","created_at":"2026-10-25T18:48:29.848Z","trace_propagation_headers":{"sentry-trace":"cafc4882dbaf4919914b706462899835-309fe4633f0b42e2","baggage":"sentry-trace_id=cafc4882dbaf4919914b706462899835,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T693Q1WZNCX0XAEMQXBGH9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.850Z","job_size_bytes":65,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-07a3642ae51f83f74c1bfddd: start","job_status":"start","queue_duration_s":0.212074,"scheduling_latency_s":0.212088}2065gitlab # [ 246.783655] gitaly[904]: time="2026-10-25T18:48:29.836Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 diskcache=068659c9-c1c3-40a7-8f7f-06dbb5e00f27 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:48:39.093" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:29.764" pid=904 ref=refs/keep-around/a786a2e66da786487d047f8106fd2d599a2fa9a2 remote_ip="2001:db8:1::1" revision=a786a2e66da786487d047f8106fd2d599a2fa9a2 span.kind=server system=grpc username=root2066gitlab # [ 246.793742] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.079Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"07a3642ae51f83f74c1bfddd","created_at":"2026-10-25T18:48:29.848Z","trace_propagation_headers":{"sentry-trace":"cafc4882dbaf4919914b706462899835-309fe4633f0b42e2","baggage":"sentry-trace_id=cafc4882dbaf4919914b706462899835,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T693Q1WZNCX0XAEMQXBGH9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.850Z","job_size_bytes":65,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-07a3642ae51f83f74c1bfddd: done: 0.016395 sec","job_status":"done","queue_duration_s":0.212074,"scheduling_latency_s":0.212088,"redis_calls":7,"redis_duration_s":0.0008719999999999999,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000231,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016395,"completed_at":"2026-10-25T18:48:30.079Z","load_balancing_strategy":"primary","db_duration_s":0.002737,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2067gitlab # [ 246.838725] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.079Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"258a967cb5532ff256d1ca94","created_at":"2026-10-25T18:48:29.851Z","trace_propagation_headers":{"sentry-trace":"cafc4882dbaf4919914b706462899835-309fe4633f0b42e2","baggage":"sentry-trace_id=cafc4882dbaf4919914b706462899835,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T693Q1WZNCX0XAEMQXBGH9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.852Z","job_size_bytes":65,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-258a967cb5532ff256d1ca94: start","job_status":"start","queue_duration_s":0.227276,"scheduling_latency_s":0.22729}2068gitlab # [ 246.847971] gitaly[904]: time="2026-10-25T18:48:29.837Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=48 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5641 command.oublock=8 command.real_time_ms=65 command.spawn_token_fork_ms=1 command.system_time_ms=32 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:48:39.093" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:29.764" grpc.time_ms=72.693 pid=904 ref=refs/keep-around/a786a2e66da786487d047f8106fd2d599a2fa9a2 remote_ip="2001:db8:1::1" revision=a786a2e66da786487d047f8106fd2d599a2fa9a2 span.kind=server system=grpc username=root2069gitlab # [ 246.890627] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.114Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"258a967cb5532ff256d1ca94","created_at":"2026-10-25T18:48:29.851Z","trace_propagation_headers":{"sentry-trace":"cafc4882dbaf4919914b706462899835-309fe4633f0b42e2","baggage":"sentry-trace_id=cafc4882dbaf4919914b706462899835,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T693Q1WZNCX0XAEMQXBGH9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.852Z","job_size_bytes":65,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-258a967cb5532ff256d1ca94: done: 0.034286 sec","job_status":"done","queue_duration_s":0.227276,"scheduling_latency_s":0.22729,"redis_calls":10,"redis_duration_s":0.001609,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000663,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034286,"completed_at":"2026-10-25T18:48:30.114Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070gitlab # [ 246.929144] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.115Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"25af7f289f95c78c6bc7c12a","created_at":"2026-10-25T18:48:29.882Z","trace_propagation_headers":{"sentry-trace":"cafc4882dbaf4919914b706462899835-309fe4633f0b42e2","baggage":"sentry-trace_id=cafc4882dbaf4919914b706462899835,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T693Q1WZNCX0XAEMQXBGH9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.885Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-25af7f289f95c78c6bc7c12a: start","job_status":"start","queue_duration_s":0.22983,"scheduling_latency_s":0.229845}2071gitlab # [ 246.929855] gitaly[904]: time="2026-10-25T18:48:29.841Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:48:59.140" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:29.840" grpc.time_ms=0.498 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2072gitlab # [ 246.930467] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.192Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"25af7f289f95c78c6bc7c12a","created_at":"2026-10-25T18:48:29.882Z","trace_propagation_headers":{"sentry-trace":"cafc4882dbaf4919914b706462899835-309fe4633f0b42e2","baggage":"sentry-trace_id=cafc4882dbaf4919914b706462899835,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M5T693Q1WZNCX0XAEMQXBGH9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:29.885Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-25af7f289f95c78c6bc7c12a: done: 0.077171 sec","job_status":"done","queue_duration_s":0.22983,"scheduling_latency_s":0.229845,"gitaly_calls":2,"gitaly_duration_s":0.004296,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004415,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000563,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000995,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000172,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000144,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000474,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000574,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055083,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.058130293,"duration_s":0.077171,"completed_at":"2026-10-25T18:48:30.192Z","load_balancing_strategy":"primary","db_duration_s":0.016968,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2073gitlab # [ 246.970741] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.193Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"3b0039782645b4896c7d53cd","created_at":"2026-10-25T18:48:30.149Z","trace_propagation_headers":{"sentry-trace":"cafc4882dbaf4919914b706462899835-190a289fa5064f24","baggage":"sentry-trace_id=cafc4882dbaf4919914b706462899835,sentry-release=Unknown"},"correlation_id":"01M5T693Q1WZNCX0XAEMQXBGH9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:30.150Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3b0039782645b4896c7d53cd: start","job_status":"start","queue_duration_s":0.042468,"scheduling_latency_s":0.042484}2074gitlab # [ 247.013638] gitaly[904]: time="2026-10-25T18:48:29.847Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:48:39.093" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:29.764" pid=904 ref=refs/keep-around/a786a2e66da786487d047f8106fd2d599a2fa9a2 remote_ip="2001:db8:1::1" revision=a786a2e66da786487d047f8106fd2d599a2fa9a2 span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2075gitlab # [ 247.014430] gitaly[904]: time="2026-10-25T18:48:29.847Z" level=info msg="optimized repository" clean_stale_data_time=15.61651ms clean_work_trees_time="69.283µs" commit_graph_time="3.631µs" correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.912µs" pid=904 prune_time="3.632µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.749µs" storage_name=default system=housekeeping total_time=15.794186ms2076gitlab # [ 247.014930] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:30.245Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"3b0039782645b4896c7d53cd","created_at":"2026-10-25T18:48:30.149Z","trace_propagation_headers":{"sentry-trace":"cafc4882dbaf4919914b706462899835-190a289fa5064f24","baggage":"sentry-trace_id=cafc4882dbaf4919914b706462899835,sentry-release=Unknown"},"correlation_id":"01M5T693Q1WZNCX0XAEMQXBGH9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:30.150Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3b0039782645b4896c7d53cd: done: 0.051845 sec","job_status":"done","queue_duration_s":0.042468,"scheduling_latency_s":0.042484,"redis_calls":7,"redis_duration_s":0.02596,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.025139,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051845,"completed_at":"2026-10-25T18:48:30.245Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2077gitlab # [ 247.068647] gitaly[904]: time="2026-10-25T18:48:29.847Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-25T18:48:39.093" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-25T18:48:29.764" pid=904 ref=refs/keep-around/a786a2e66da786487d047f8106fd2d599a2fa9a2 remote_ip="2001:db8:1::1" revision=a786a2e66da786487d047f8106fd2d599a2fa9a2 span.kind=server system=grpc username=root2078gitlab # [ 247.069422] gitaly[904]: time="2026-10-25T18:48:30.161Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-25T18:49:00.159" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:30.160" grpc.time_ms=0.612 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2079gitlab # [ 247.070098] gitaly[904]: time="2026-10-25T18:48:30.182Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T693Q1WZNCX0XAEMQXBGH9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:49:00.182" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:30.182" grpc.time_ms=0.4 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2080gitlab # [ 247.072700] gitaly[904]: time="2026-10-25T18:48:30.641Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T694MHD9VVBWHVKT9VK4AT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:49:00.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:30.640" grpc.time_ms=0.44 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2081gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses2082gitlab # [ 247.636496] gitaly[904]: time="2026-10-25T18:48:31.771Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T695PWX1RRN60J10FDW92Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:49:01.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:31.770" grpc.time_ms=0.68 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2083gitlab # [ 247.680486] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1062084gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses, in 0.16 seconds)2085gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses2086gitlab # [ 248.809379] gitaly[904]: time="2026-10-25T18:48:32.944Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T696TQY2D4TAATJ2EWSQR6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:49:02.242" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:32.943" grpc.time_ms=0.772 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2087gitlab # [ 248.838416] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1172088gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses, in 0.16 seconds)2089gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses2090gitlab # [ 249.900597] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:34.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"058f44900e9bff4eee817dd5","created_at":"2026-10-25T18:48:34.020Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e77b0c82061149ec10513db24d811ab6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.032Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-058f44900e9bff4eee817dd5: start","job_status":"start","queue_duration_s":0.002222,"scheduling_latency_s":0.002268}2091gitlab # [ 249.937504] gitaly[904]: time="2026-10-25T18:48:34.072Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T697YXKRZBM4GS3Z4PF048 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:49:04.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:34.071" grpc.time_ms=0.857 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2092gitlab # [ 249.960979] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 812093gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses, in 0.12 seconds)2094gitlab # [ 250.569403] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:34.704Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"058f44900e9bff4eee817dd5","created_at":"2026-10-25T18:48:34.020Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e77b0c82061149ec10513db24d811ab6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.032Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-058f44900e9bff4eee817dd5: done: 0.669167 sec","job_status":"done","queue_duration_s":0.002222,"scheduling_latency_s":0.002268,"redis_calls":13,"redis_duration_s":0.009179,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001779,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00383,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00357,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.133,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.133,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.270502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000830832,"exclusive_lock_hold_duration_s":0.517980358,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.669167,"completed_at":"2026-10-25T18:48:34.704Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.295051,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2095gitlab # [ 250.617418] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:34.752Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c5865c1f4dcedf794088cf9d","created_at":"2026-10-25T18:48:34.097Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd33517974eb352ef018ac0eebe28ac7","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.110Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c5865c1f4dcedf794088cf9d: start","job_status":"start","queue_duration_s":0.642442,"scheduling_latency_s":0.642492}2096gitlab # [ 250.680959] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:34.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c5865c1f4dcedf794088cf9d","created_at":"2026-10-25T18:48:34.097Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd33517974eb352ef018ac0eebe28ac7","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.110Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c5865c1f4dcedf794088cf9d: done: 0.063688 sec","job_status":"done","queue_duration_s":0.642442,"scheduling_latency_s":0.642492,"redis_calls":11,"redis_duration_s":0.012836,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00105,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007642,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004144,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000735847,"exclusive_lock_hold_duration_s":0.004060572,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.063688,"completed_at":"2026-10-25T18:48:34.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002847,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2097gitlab # [ 250.706903] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:34.820Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"295435f0486833707788c884","created_at":"2026-10-25T18:48:34.117Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53d9347f9cc194dbcc4432acb4d14c19","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.158Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-295435f0486833707788c884: start","job_status":"start","queue_duration_s":0.662336,"scheduling_latency_s":0.662372}2098gitlab # [ 250.743506] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:34.878Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"295435f0486833707788c884","created_at":"2026-10-25T18:48:34.117Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53d9347f9cc194dbcc4432acb4d14c19","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.158Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-295435f0486833707788c884: done: 0.058071 sec","job_status":"done","queue_duration_s":0.662336,"scheduling_latency_s":0.662372,"redis_calls":12,"redis_duration_s":0.01356,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003379,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005967,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004214,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000955149,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.058071,"completed_at":"2026-10-25T18:48:34.878Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001518,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2099gitlab # [ 250.764833] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:34.883Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ae0eb2e603efc9ed090f694f","created_at":"2026-10-25T18:48:34.172Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"a48e66591bf1050b5102485195465ca0","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.183Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ae0eb2e603efc9ed090f694f: start","job_status":"start","queue_duration_s":0.699563,"scheduling_latency_s":0.699591}2100gitlab # [ 250.807165] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:34.942Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ae0eb2e603efc9ed090f694f","created_at":"2026-10-25T18:48:34.172Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"a48e66591bf1050b5102485195465ca0","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.183Z","job_size_bytes":3,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ae0eb2e603efc9ed090f694f: done: 0.058793 sec","job_status":"done","queue_duration_s":0.699563,"scheduling_latency_s":0.699591,"redis_calls":10,"redis_duration_s":0.0032329999999999998,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000627,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001862,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000744,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000532191,"duration_s":0.058793,"completed_at":"2026-10-25T18:48:34.942Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000804,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2101gitlab # [ 250.847347] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:34.982Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e6fea4729754c59b865e8b6a","created_at":"2026-10-25T18:48:34.202Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87cde3bb7da9f564dd2b2ead89b7bb75","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.216Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e6fea4729754c59b865e8b6a: start","job_status":"start","queue_duration_s":0.766419,"scheduling_latency_s":0.766448}2102gitlab # [ 250.919992] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.054Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e6fea4729754c59b865e8b6a","created_at":"2026-10-25T18:48:34.202Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87cde3bb7da9f564dd2b2ead89b7bb75","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.216Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e6fea4729754c59b865e8b6a: done: 0.071951 sec","job_status":"done","queue_duration_s":0.766419,"scheduling_latency_s":0.766448,"redis_calls":9,"redis_duration_s":0.006122,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0014,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002436,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002286,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.071951,"completed_at":"2026-10-25T18:48:35.054Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2103gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses2104gitlab # [ 250.922565] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3d7f37c3492fe41d5493ad27","created_at":"2026-10-25T18:48:34.256Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20b20f2927333b546cb18bf4122ab1f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.274Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3d7f37c3492fe41d5493ad27: start","job_status":"start","queue_duration_s":0.781844,"scheduling_latency_s":0.781876}2105gitlab # [ 250.955687] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.090Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3d7f37c3492fe41d5493ad27","created_at":"2026-10-25T18:48:34.256Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20b20f2927333b546cb18bf4122ab1f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.274Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3d7f37c3492fe41d5493ad27: done: 0.034318 sec","job_status":"done","queue_duration_s":0.781844,"scheduling_latency_s":0.781876,"redis_calls":12,"redis_duration_s":0.003685,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000626,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001728,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009267658,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.034318,"completed_at":"2026-10-25T18:48:35.090Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003839,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2106gitlab # [ 251.015591] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"9264c1c4772aa0afd8f41592","created_at":"2026-10-25T18:48:34.285Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"793ce805fc35d6d8f605a800728ac61b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.297Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-9264c1c4772aa0afd8f41592: start","job_status":"start","queue_duration_s":0.795419,"scheduling_latency_s":0.795441}2107gitlab # [ 251.016256] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.100Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2108gitlab # [ 251.016735] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.104Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"9264c1c4772aa0afd8f41592","created_at":"2026-10-25T18:48:34.285Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"793ce805fc35d6d8f605a800728ac61b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.297Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-9264c1c4772aa0afd8f41592: done: 0.011572 sec","job_status":"done","queue_duration_s":0.795419,"scheduling_latency_s":0.795441,"redis_calls":9,"redis_duration_s":0.001975,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006237,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011572,"completed_at":"2026-10-25T18:48:35.104Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000407,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2109gitlab # [ 251.032517] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.105Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8739e809d85d273dc38688b1","created_at":"2026-10-25T18:48:34.317Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5b4d1869292f025b83c57cb23b260b7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.331Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8739e809d85d273dc38688b1: start","job_status":"start","queue_duration_s":0.774265,"scheduling_latency_s":0.774279}2110gitlab # [ 251.045108] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8739e809d85d273dc38688b1","created_at":"2026-10-25T18:48:34.317Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5b4d1869292f025b83c57cb23b260b7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.331Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8739e809d85d273dc38688b1: done: 0.028161 sec","job_status":"done","queue_duration_s":0.774265,"scheduling_latency_s":0.774279,"redis_calls":11,"redis_duration_s":0.006213,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001186,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000372673,"exclusive_lock_hold_duration_s":0.007408204,"duration_s":0.028161,"completed_at":"2026-10-25T18:48:35.133Z","load_balancing_strategy":"primary","db_duration_s":0.00514,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2111gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses, in 0.17 seconds)2112gitlab # [ 251.087549] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b4d01e91b9cf94f4a48dc8e7","created_at":"2026-10-25T18:48:34.355Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f651c7b047bdb5a7705a75ab459b109","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.368Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b4d01e91b9cf94f4a48dc8e7: start","job_status":"start","queue_duration_s":0.767013,"scheduling_latency_s":0.76704}2113gitlab # [ 251.129511] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b4d01e91b9cf94f4a48dc8e7","created_at":"2026-10-25T18:48:34.355Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f651c7b047bdb5a7705a75ab459b109","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.368Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b4d01e91b9cf94f4a48dc8e7: done: 0.04765 sec","job_status":"done","queue_duration_s":0.767013,"scheduling_latency_s":0.76704,"redis_calls":11,"redis_duration_s":0.0037300000000000002,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000243,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001961,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001526,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000626057,"exclusive_lock_hold_duration_s":0.022005311,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.04765,"completed_at":"2026-10-25T18:48:35.182Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018843,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2114gitlab # [ 251.133995] nix-daemon[2801]: accepted connection from pid 3710, user nixbot2115gitlab # [ 251.134804] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 842116gitlab # [ 251.136838] gitaly[904]: time="2026-10-25T18:48:35.246Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T69938J8NFK7E3MM93HG1W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:49:05.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:35.245" grpc.time_ms=0.528 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2117gitlab # [ 251.179121] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e2fc7b6a997ef2ab11c80d4e","created_at":"2026-10-25T18:48:34.381Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a85adc225cabcc87f7c712aa8b0e8a68","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.395Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e2fc7b6a997ef2ab11c80d4e: start","job_status":"start","queue_duration_s":0.795558,"scheduling_latency_s":0.795581}2118gitlab # [ 251.179825] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e2fc7b6a997ef2ab11c80d4e","created_at":"2026-10-25T18:48:34.381Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a85adc225cabcc87f7c712aa8b0e8a68","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.395Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e2fc7b6a997ef2ab11c80d4e: done: 0.03934 sec","job_status":"done","queue_duration_s":0.795558,"scheduling_latency_s":0.795581,"redis_calls":9,"redis_duration_s":0.0222,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000349,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001511,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.02034,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03934,"completed_at":"2026-10-25T18:48:35.230Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001754,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119gitlab # [ 251.228265] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.231Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"2d3da82f080df0bd8d17baa3","created_at":"2026-10-25T18:48:34.412Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3d8a20c72b5362d6ea0b07f5869a937","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.425Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-2d3da82f080df0bd8d17baa3: start","job_status":"start","queue_duration_s":0.805921,"scheduling_latency_s":0.805963}2120gitlab # [ 251.228520] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.242Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"2d3da82f080df0bd8d17baa3","created_at":"2026-10-25T18:48:34.412Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3d8a20c72b5362d6ea0b07f5869a937","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.425Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-2d3da82f080df0bd8d17baa3: done: 0.011155 sec","job_status":"done","queue_duration_s":0.805921,"scheduling_latency_s":0.805963,"redis_calls":12,"redis_duration_s":0.001596,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000352,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000297,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000192762,"duration_s":0.011155,"completed_at":"2026-10-25T18:48:35.242Z","load_balancing_strategy":"primary","db_duration_s":0.000308,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121gitlab # [ 251.247189] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.243Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"9da8a36407d9ccf72e89eb8e","created_at":"2026-10-25T18:48:34.452Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16301cf8c20e68d7915069bdcd92bc5e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.466Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-9da8a36407d9ccf72e89eb8e: start","job_status":"start","queue_duration_s":0.777602,"scheduling_latency_s":0.777615}2122gitlab # [ 251.289915] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"9da8a36407d9ccf72e89eb8e","created_at":"2026-10-25T18:48:34.452Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16301cf8c20e68d7915069bdcd92bc5e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.466Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-9da8a36407d9ccf72e89eb8e: done: 0.0149 sec","job_status":"done","queue_duration_s":0.777602,"scheduling_latency_s":0.777615,"redis_calls":11,"redis_duration_s":0.0017,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000641,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0149,"completed_at":"2026-10-25T18:48:35.258Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000675,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2123gitlab # [ 251.296805] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.259Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"92bebd9dfac475cc6cdc8aaa","created_at":"2026-10-25T18:48:34.482Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0687ea0fa04a46fb7d24dfa68a74e25e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.495Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-92bebd9dfac475cc6cdc8aaa: start","job_status":"start","queue_duration_s":0.764507,"scheduling_latency_s":0.764526}2124gitlab # [ 251.308978] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"92bebd9dfac475cc6cdc8aaa","created_at":"2026-10-25T18:48:34.482Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0687ea0fa04a46fb7d24dfa68a74e25e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.495Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-92bebd9dfac475cc6cdc8aaa: done: 0.095345 sec","job_status":"done","queue_duration_s":0.764507,"scheduling_latency_s":0.764526,"redis_calls":65,"redis_duration_s":0.008378,"redis_read_bytes":114,"redis_write_bytes":41332,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000506,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.000434,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37035,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.006976,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214273,"duration_s":0.095345,"completed_at":"2026-10-25T18:48:35.355Z","load_balancing_strategy":"primary","db_duration_s":0.0005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125gitlab # [ 251.356966] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ef6327cee7f181446590deef","created_at":"2026-10-25T18:48:34.509Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efc87ef6c925e978f1002545156fa152","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-25T18:48:34.522Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ef6327cee7f181446590deef: start","job_status":"start","queue_duration_s":0.833255,"scheduling_latency_s":0.833269}2126gitlab # [ 251.358743] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.363Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2127gitlab # [ 251.398793] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ef6327cee7f181446590deef","created_at":"2026-10-25T18:48:34.509Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efc87ef6c925e978f1002545156fa152","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-25T18:48:34.522Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ef6327cee7f181446590deef: done: 0.012008 sec","job_status":"done","queue_duration_s":0.833255,"scheduling_latency_s":0.833269,"redis_calls":9,"redis_duration_s":0.001104,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012008,"completed_at":"2026-10-25T18:48:35.367Z","load_balancing_strategy":"primary","db_duration_s":0.000431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2128gitlab # [ 251.400023] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3509b2e965d7b9f627c46ff0","created_at":"2026-10-25T18:48:34.542Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf938d40dc34cad1a26c30ab15b743ad","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.554Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3509b2e965d7b9f627c46ff0: start","job_status":"start","queue_duration_s":0.814138,"scheduling_latency_s":0.814151}2129gitlab # [ 251.452453] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.587Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3509b2e965d7b9f627c46ff0","created_at":"2026-10-25T18:48:34.542Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf938d40dc34cad1a26c30ab15b743ad","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.554Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3509b2e965d7b9f627c46ff0: done: 0.219129 sec","job_status":"done","queue_duration_s":0.814138,"scheduling_latency_s":0.814151,"redis_calls":564,"redis_duration_s":0.060513000000000004,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.059587,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.184124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.219129,"completed_at":"2026-10-25T18:48:35.587Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2130gitlab # [ 251.458375] nix-daemon[2801]: accepted connection from pid 3664, user nixbot2131gitlab # [ 251.492438] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fa9c2371edd106d1897c52cd","created_at":"2026-10-25T18:48:34.575Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab29cc1c545ab0cf4f36b7979da161cb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.589Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fa9c2371edd106d1897c52cd: start","job_status":"start","queue_duration_s":1.038836,"scheduling_latency_s":1.038867}2132gitlab # [ 251.515537] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.651Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2133gitlab # [ 251.521720] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.657Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fa9c2371edd106d1897c52cd","created_at":"2026-10-25T18:48:34.575Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab29cc1c545ab0cf4f36b7979da161cb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.589Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fa9c2371edd106d1897c52cd: done: 0.029464 sec","job_status":"done","queue_duration_s":1.038836,"scheduling_latency_s":1.038867,"redis_calls":9,"redis_duration_s":0.001882,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000806,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029464,"completed_at":"2026-10-25T18:48:35.657Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2134gitlab # [ 251.522703] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.658Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e081db9e2697235ebe6a69d5","created_at":"2026-10-25T18:48:34.606Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"782687594eda04ec348e2f6d4821d3b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.620Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e081db9e2697235ebe6a69d5: start","job_status":"start","queue_duration_s":1.037303,"scheduling_latency_s":1.037322}2135gitlab # [ 251.563720] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e081db9e2697235ebe6a69d5","created_at":"2026-10-25T18:48:34.606Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"782687594eda04ec348e2f6d4821d3b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.620Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e081db9e2697235ebe6a69d5: done: 0.010837 sec","job_status":"done","queue_duration_s":1.037303,"scheduling_latency_s":1.037322,"redis_calls":9,"redis_duration_s":0.001062,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00023,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010837,"completed_at":"2026-10-25T18:48:35.669Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136gitlab # [ 251.609047] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"72820345c03371c9bf3b5a66","created_at":"2026-10-25T18:48:34.640Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1082468ab8592100e78a9167b323f0cb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.651Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-72820345c03371c9bf3b5a66: start","job_status":"start","queue_duration_s":1.018525,"scheduling_latency_s":1.018538}2137gitlab # [ 251.609754] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.676Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2138gitlab # [ 251.610249] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.680Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"72820345c03371c9bf3b5a66","created_at":"2026-10-25T18:48:34.640Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1082468ab8592100e78a9167b323f0cb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.651Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-72820345c03371c9bf3b5a66: done: 0.010208 sec","job_status":"done","queue_duration_s":1.018525,"scheduling_latency_s":1.018538,"redis_calls":9,"redis_duration_s":0.001098,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010208,"completed_at":"2026-10-25T18:48:35.680Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139gitlab # [ 251.650693] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.680Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"c8a9da390738c55c9083b006","created_at":"2026-10-25T18:48:34.698Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ec79862e107e4abe64f9b60eb2bf8f7","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.750Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-c8a9da390738c55c9083b006: start","job_status":"start","queue_duration_s":0.929873,"scheduling_latency_s":0.929886}2140gitlab # [ 251.651601] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"c8a9da390738c55c9083b006","created_at":"2026-10-25T18:48:34.698Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ec79862e107e4abe64f9b60eb2bf8f7","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.750Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-c8a9da390738c55c9083b006: done: 0.016747 sec","job_status":"done","queue_duration_s":0.929873,"scheduling_latency_s":0.929886,"redis_calls":9,"redis_duration_s":0.001086,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000227,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.016747,"completed_at":"2026-10-25T18:48:35.697Z","load_balancing_strategy":"primary","db_duration_s":0.004703,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141gitlab # [ 251.707874] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.698Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c4c03f5100e8101b28a9de47","created_at":"2026-10-25T18:48:34.772Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c564bd218c3af24464af6d1788a221e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.787Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c4c03f5100e8101b28a9de47: start","job_status":"start","queue_duration_s":0.910516,"scheduling_latency_s":0.910529}2142gitlab # [ 251.708600] gitaly[904]: time="2026-10-25T18:48:35.760Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T699MBHF4J2CP2RNHZYA4P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:49:05.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:35.759" grpc.time_ms=0.71 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2143gitlab # [ 251.709223] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 942144gitlab # [ 251.709703] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.766Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c4c03f5100e8101b28a9de47","created_at":"2026-10-25T18:48:34.772Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c564bd218c3af24464af6d1788a221e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.787Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c4c03f5100e8101b28a9de47: done: 0.067947 sec","job_status":"done","queue_duration_s":0.910516,"scheduling_latency_s":0.910529,"redis_calls":70,"redis_duration_s":0.013713,"redis_read_bytes":119,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001216,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00044,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.011604,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.067947,"completed_at":"2026-10-25T18:48:35.766Z","load_balancing_strategy":"primary","db_duration_s":0.001177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 251.769881] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.767Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3e5d349bdbdfb31119fa4ba3","created_at":"2026-10-25T18:48:34.799Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f00fa926fb1c677cc8605ef76fb7df98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.813Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3e5d349bdbdfb31119fa4ba3: start","job_status":"start","queue_duration_s":0.953441,"scheduling_latency_s":0.953462}2146gitlab # [ 251.770542] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.774Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2147gitlab # [ 251.771074] gitaly[904]: time="2026-10-25T18:48:35.783Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M5T699MBHF4J2CP2RNHZYA4P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:45.108" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:35.778" grpc.time_ms=5.485 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2148gitlab # [ 251.771729] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3e5d349bdbdfb31119fa4ba3","created_at":"2026-10-25T18:48:34.799Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f00fa926fb1c677cc8605ef76fb7df98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.813Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3e5d349bdbdfb31119fa4ba3: done: 0.011575 sec","job_status":"done","queue_duration_s":0.953441,"scheduling_latency_s":0.953462,"redis_calls":9,"redis_duration_s":0.0011480000000000001,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011575,"completed_at":"2026-10-25T18:48:35.778Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000298,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2149gitlab # [ 251.807328] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.779Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c6bca2f739e33f3226e63435","created_at":"2026-10-25T18:48:34.821Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f8d4fb15685d9ab033061d5276dbce2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-25T18:48:34.832Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c6bca2f739e33f3226e63435: start","job_status":"start","queue_duration_s":0.946872,"scheduling_latency_s":0.946886}2150gitlab # [ 251.819664] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.791Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2151gitlab # [ 251.822883] gitaly[904]: time="2026-10-25T18:48:35.866Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T699QGBDCSB13CF76CKV6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:49:05.263" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:35.864" grpc.time_ms=1.659 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2152gitlab # [ 251.834579] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1032153gitlab # [ 251.836693] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c6bca2f739e33f3226e63435","created_at":"2026-10-25T18:48:34.821Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f8d4fb15685d9ab033061d5276dbce2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-25T18:48:34.832Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c6bca2f739e33f3226e63435: done: 0.01554 sec","job_status":"done","queue_duration_s":0.946872,"scheduling_latency_s":0.946886,"redis_calls":9,"redis_duration_s":0.004045,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002997,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01554,"completed_at":"2026-10-25T18:48:35.795Z","load_balancing_strategy":"primary","db_duration_s":0.000397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2154gitlab # [ 251.878989] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b893f2e17689ab8774dbe198","created_at":"2026-10-25T18:48:34.847Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2244166c5c48e5ddb3b15cf4ce73b1b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-25T18:48:34.860Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b893f2e17689ab8774dbe198: start","job_status":"start","queue_duration_s":0.935602,"scheduling_latency_s":0.935616}2155gitlab # [ 251.884761] gitaly[904]: time="2026-10-25T18:48:35.894Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M5T699QGBDCSB13CF76CKV6M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:45.037" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:35.887" grpc.time_ms=6.553 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2156gitlab # [ 251.931168] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b893f2e17689ab8774dbe198","created_at":"2026-10-25T18:48:34.847Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2244166c5c48e5ddb3b15cf4ce73b1b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-25T18:48:34.860Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b893f2e17689ab8774dbe198: done: 0.020655 sec","job_status":"done","queue_duration_s":0.935602,"scheduling_latency_s":0.935616,"redis_calls":9,"redis_duration_s":0.001265,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020655,"completed_at":"2026-10-25T18:48:35.816Z","load_balancing_strategy":"primary","db_duration_s":0.00263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2157gitlab # [ 251.932887] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.817Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"49665acd29a6f89e02fb6546","created_at":"2026-10-25T18:48:34.870Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a28a492e9f2371feb3798a3fce6a93a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.882Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-49665acd29a6f89e02fb6546: start","job_status":"start","queue_duration_s":0.935708,"scheduling_latency_s":0.935726}2158gitlab # [ 251.945161] gitaly[904]: time="2026-10-25T18:48:36.040Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M5T699V4XH367XGRRJJ52YGC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-25T18:49:06.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-25T18:48:36.039" grpc.time_ms=0.534 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2159gitlab # [ 251.956863] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"49665acd29a6f89e02fb6546","created_at":"2026-10-25T18:48:34.870Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a28a492e9f2371feb3798a3fce6a93a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.882Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-49665acd29a6f89e02fb6546: done: 0.023041 sec","job_status":"done","queue_duration_s":0.935708,"scheduling_latency_s":0.935726,"redis_calls":18,"redis_duration_s":0.002613,"redis_read_bytes":21,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000624,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000166,"redis_queues_read_bytes":4,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023041,"completed_at":"2026-10-25T18:48:35.840Z","load_balancing_strategy":"primary","db_duration_s":0.001201,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160gitlab # [ 252.028835] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"cf5468c26fc3bc604c809cab","created_at":"2026-10-25T18:48:34.908Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e982605eada278e7f9ad7c967d05d927","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.940Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-cf5468c26fc3bc604c809cab: start","job_status":"start","queue_duration_s":0.901692,"scheduling_latency_s":0.901713}2161gitlab # [ 252.029754] gitaly[904]: time="2026-10-25T18:48:36.070Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M5T699V4XH367XGRRJJ52YGC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-25T18:48:46.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-25T18:48:36.065" grpc.time_ms=5.835 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2162gitlab # [ 252.030505] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/10/25:18:48:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a786a2e66da786487d047f8106fd2d599a2fa9a2 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1662163gitlab # [ 252.030964] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.857Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"cf5468c26fc3bc604c809cab","created_at":"2026-10-25T18:48:34.908Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e982605eada278e7f9ad7c967d05d927","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:34.940Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-cf5468c26fc3bc604c809cab: done: 0.015678 sec","job_status":"done","queue_duration_s":0.901692,"scheduling_latency_s":0.901713,"redis_calls":10,"redis_duration_s":0.001256,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215949,"duration_s":0.015678,"completed_at":"2026-10-25T18:48:35.857Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001783,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164gitlab # [ 252.078741] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ff18bedaadfca16c6785bfb9","created_at":"2026-10-25T18:48:34.997Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81900efa1d90eb97727e060600cff347","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:35.011Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ff18bedaadfca16c6785bfb9: start","job_status":"start","queue_duration_s":0.847143,"scheduling_latency_s":0.847156}2165gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses2166gitlab # [ 252.079414] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.886Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ff18bedaadfca16c6785bfb9","created_at":"2026-10-25T18:48:34.997Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81900efa1d90eb97727e060600cff347","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:35.011Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ff18bedaadfca16c6785bfb9: done: 0.028142 sec","job_status":"done","queue_duration_s":0.847143,"scheduling_latency_s":0.847156,"redis_calls":13,"redis_duration_s":0.00201,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000587,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000697,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000383289,"exclusive_lock_hold_duration_s":0.012170262,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.028142,"completed_at":"2026-10-25T18:48:35.886Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2167gitlab # [ 252.133655] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.887Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"dce2d9bc3ef06a74f7e63ed3","created_at":"2026-10-25T18:48:35.026Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ffeed81e7647e89328d8440612c0b75","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:35.037Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-dce2d9bc3ef06a74f7e63ed3: start","job_status":"start","queue_duration_s":0.850834,"scheduling_latency_s":0.850848}2168gitlab # [ 252.134515] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.905Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"dce2d9bc3ef06a74f7e63ed3","created_at":"2026-10-25T18:48:35.026Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-e89c4dd389bb4597","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ffeed81e7647e89328d8440612c0b75","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-25T18:48:35.037Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-dce2d9bc3ef06a74f7e63ed3: done: 0.01714 sec","job_status":"done","queue_duration_s":0.850834,"scheduling_latency_s":0.850848,"redis_calls":12,"redis_duration_s":0.00195,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000335,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000663,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000357029,"exclusive_lock_hold_duration_s":0.002974121,"duration_s":0.01714,"completed_at":"2026-10-25T18:48:35.905Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001521,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2169gitlab # [ 252.185639] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bedcfb3bf644d91405287780","created_at":"2026-10-25T18:48:35.269Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-f3415fb1bbf74f5f","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0687ea0fa04a46fb7d24dfa68a74e25e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:48:35.349Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bedcfb3bf644d91405287780: start","job_status":"start","queue_duration_s":0.556421,"scheduling_latency_s":0.556435}2170gitlab # [ 252.186783] nixbot[2636]: {"ts": "2026-10-25T18:48:36.258Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 2, "sha": "a786a2e66da786487d047f8106fd2d599a2fa9a2", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}2171gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a786a2e66da786487d047f8106fd2d599a2fa9a2/statuses, in 0.10 seconds)2172(finished: subtest: eval and build post commit statuses, in 5.77 seconds)2173(finished: run the VM test script, in 303.98 seconds)2174test script finished in 304.01s2175cleanup2176kill QemuMachine (pid 46)2177gitlab # [ 252.187259] sidekiq-cluster[2508]: {"severity":"INFO","time":"2026-10-25T18:48:35.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bedcfb3bf644d91405287780","created_at":"2026-10-25T18:48:35.269Z","trace_propagation_headers":{"sentry-trace":"335180c2a91a444e870667ee04f779bd-f3415fb1bbf74f5f","baggage":"sentry-trace_id=335180c2a91a444e870667ee04f779bd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"0687ea0fa04a46fb7d24dfa68a74e25e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-25T18:48:35.349Z","job_size_bytes":2,"pid":2508,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bedcfb3bf644d91405287780: done: 0.025674 sec","job_status":"done","queue_duration_s":0.556421,"scheduling_latency_s":0.556435,"redis_calls":13,"redis_duration_s":0.004536,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003393,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2178gitlab # s2179(finished: cleanup, in 1.44 seconds)2180additionally exposed symbols:2181 gitlab,2182 vlan1,2183 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2184[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-25T18:47:43.616Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2185[]2186[]2187[]2188[]2189[]2190[]2191[]2192[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'a786a2e66da786487d047f8106fd2d599a2fa9a2', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': [{'level': 'warning', 'message': "unable to download 'https://cache.nixos.org/…': Could not resolve hostname (6) Could not resolve host: cache.nixos.org", 'count': 1}], 'error': None, 'created_at': '2026-10-25T18:48:29.660394Z', 'started_at': '2026-10-25T18:48:29.905405Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2193[{'id': 5, 'sha': 'a786a2e66da786487d047f8106fd2d599a2fa9a2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-25T18:48:29.877Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2194[{'id': 5, 'sha': 'a786a2e66da786487d047f8106fd2d599a2fa9a2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-25T18:48:29.877Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2195[{'id': 5, 'sha': 'a786a2e66da786487d047f8106fd2d599a2fa9a2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-25T18:48:29.877Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2196[{'id': 5, 'sha': 'a786a2e66da786487d047f8106fd2d599a2fa9a2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-25T18:48:29.877Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2197[{'id': 5, 'sha': 'a786a2e66da786487d047f8106fd2d599a2fa9a2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-25T18:48:29.877Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2198[{'id': 5, 'sha': 'a786a2e66da786487d047f8106fd2d599a2fa9a2', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-25T18:48:29.877Z', 'started_at': None, 'finished_at': '2026-10-25T18:48:35.799Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'a786a2e66da786487d047f8106fd2d599a2fa9a2', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-10-25T18:48:35.907Z', 'started_at': None, 'finished_at': '2026-10-25T18:48:36.088Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]