vm-test-run-nixbot-gitlab
x86_64-linux.nixbot-gitlab
· build #100
· raw
1Machine state will be reset. To keep it, pass --keep-machine-state2start all VLans3(finished: start all VLans, in 0.00 seconds)4Test will time out and terminate in 3600 seconds5run the VM test script6additionally exposed symbols:7 gitlab,8 vlan1,9 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh10start all VMs11gitlab: starting vm12gitlab: QEMU running (pid 13)13gitlab # Disk image does not exist, creating the virtualisation disk image...14gitlab # Formatting '/build/vm-state-gitlab/tmp.u7D3Hr4lFd', fmt=raw size=107374182415gitlab # mke2fs 1.47.3 (8-Jul-2025)16gitlab # Discarding device blocks: 0/262144 done17gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes18gitlab # Filesystem UUID: c8d7737a-61ed-43cc-9422-826f5aa5bee519gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30(finished: start all VMs, in 34.77 seconds)31gitlab: waiting for unit setup-gitlab.service32gitlab: waiting for the VM to finish booting33gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)34gitlab # 35gitlab # 36gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0037gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...38gitlab # 39gitlab # 40gitlab # 41gitlab # 42gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0043gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...44gitlab # 45gitlab # 46gitlab # Booting from ROM...47gitlab # Probing EDD (edd=off to disable)... ok48gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 202649gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/34xs09d1jkfapqaxyzjrxrgwkmr28bas-nixos-system-gitlab-test/init regInfo=/nix/store/da4xjbgkzgk5sf66qh8ffq1hq2hh8ymz-closure-info/registration console=ttyS0,115200n8 console=tty050gitlab # [ 0.000000] BIOS-provided physical RAM map:51gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable52gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved56gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved57gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved58gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable59gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved60gitlab # [ 0.000000] NX (Execute Disable) protection: active61gitlab # [ 0.000000] APIC: Static calls initialized62gitlab # [ 0.000000] SMBIOS 2.8 present.63gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201464gitlab # [ 0.000000] DMI: Memory slots populated: 1/165gitlab # [ 0.000000] Hypervisor detected: KVM66gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000067gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0068gitlab # [ 0.000001] kvm-clock: using sched offset of 503062227 cycles69gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns70gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor71gitlab # [ 0.000815] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000072gitlab # [ 0.000853] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs73gitlab # [ 0.000856] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT74gitlab # [ 0.000908] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000075gitlab # [ 0.002760] found SMP MP-table at [mem 0x000f5470-0x000f547f]76gitlab # [ 0.002770] Using GB pages for direct mapping77gitlab # [ 0.002905] RAMDISK: [mem 0xbe4e0000-0xbffcffff]78gitlab # [ 0.002913] ACPI: Early table checksum verification disabled79gitlab # [ 0.002915] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )80gitlab # [ 0.002919] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002923] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002929] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002931] ACPI: FACS 0x00000000BFFE0000 00004084gitlab # [ 0.002933] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)85gitlab # [ 0.002935] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)86gitlab # [ 0.002936] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)87gitlab # [ 0.002938] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]88gitlab # [ 0.002939] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]89gitlab # [ 0.002939] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]90gitlab # [ 0.002940] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]91gitlab # [ 0.002940] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]92gitlab # [ 0.002941] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]93gitlab # [ 0.003500] No NUMA configuration found94gitlab # [ 0.003502] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]95gitlab # [ 0.003505] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]96gitlab # [ 0.006251] Zone ranges:97gitlab # [ 0.006252] DMA [mem 0x0000000000001000-0x0000000000ffffff]98gitlab # [ 0.006254] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]99gitlab # [ 0.006256] Normal [mem 0x0000000100000000-0x00000001bfffffff]100gitlab # [ 0.006257] Device empty101gitlab # [ 0.006257] Movable zone start for each node102gitlab # [ 0.006258] Early memory node ranges103gitlab # [ 0.006258] node 0: [mem 0x0000000000001000-0x000000000009efff]104gitlab # [ 0.006260] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]105gitlab # [ 0.006260] node 0: [mem 0x0000000100000000-0x00000001bfffffff]106gitlab # [ 0.006262] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]107gitlab # [ 0.006282] On node 0, zone DMA: 1 pages in unavailable ranges108gitlab # [ 0.006568] On node 0, zone DMA: 97 pages in unavailable ranges109gitlab # [ 0.120766] On node 0, zone Normal: 40 pages in unavailable ranges110gitlab # [ 0.121800] ACPI: PM-Timer IO Port: 0x608111gitlab # [ 0.121818] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])112gitlab # [ 0.121855] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23113gitlab # [ 0.121858] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)114gitlab # [ 0.121860] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)115gitlab # [ 0.121861] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)116gitlab # [ 0.121863] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)117gitlab # [ 0.121863] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)118gitlab # [ 0.121866] ACPI: Using ACPI (MADT) for SMP configuration information119gitlab # [ 0.121867] ACPI: HPET id: 0x8086a201 base: 0xfed00000120gitlab # [ 0.121875] TSC deadline timer available121gitlab # [ 0.121879] CPU topo: Max. logical packages: 1122gitlab # [ 0.121880] CPU topo: Max. logical dies: 1123gitlab # [ 0.121881] CPU topo: Max. dies per package: 1124gitlab # [ 0.121884] CPU topo: Max. threads per core: 1125gitlab # [ 0.121885] CPU topo: Num. cores per package: 4126gitlab # [ 0.121885] CPU topo: Num. threads per package: 4127gitlab # [ 0.121885] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs128gitlab # [ 0.121913] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()129gitlab # [ 0.121933] kvm-guest: KVM setup pv remote TLB flush130gitlab # [ 0.121936] kvm-guest: setup PV sched yield131gitlab # [ 0.121957] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]132gitlab # [ 0.121959] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]133gitlab # [ 0.121960] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]134gitlab # [ 0.121962] [mem 0xc0000000-0xfeffbfff] available for PCI devices135gitlab # [ 0.121963] Booting paravirtualized kernel on KVM136gitlab # [ 0.121967] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns137gitlab # [ 0.126452] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1138gitlab # [ 0.128830] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288139gitlab # [ 0.128880] kvm-guest: PV spinlocks enabled140gitlab # [ 0.128881] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)141gitlab # [ 0.128885] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/34xs09d1jkfapqaxyzjrxrgwkmr28bas-nixos-system-gitlab-test/init regInfo=/nix/store/da4xjbgkzgk5sf66qh8ffq1hq2hh8ymz-closure-info/registration console=ttyS0,115200n8 console=tty0142gitlab # [ 0.128985] Unknown kernel command line parameters "regInfo=/nix/store/da4xjbgkzgk5sf66qh8ffq1hq2hh8ymz-closure-info/registration", will be passed to user space.143gitlab # [ 0.128998] random: crng init done144gitlab # [ 0.128998] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes145gitlab # [ 0.138428] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)146gitlab # [ 0.143251] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)147gitlab # [ 0.143330] software IO TLB: area num 4.148gitlab # [ 0.222817] Fallback order for Node 0: 0149gitlab # [ 0.222825] Built 1 zonelists, mobility grouping on. Total pages: 1572726150gitlab # [ 0.222828] Policy zone: Normal151gitlab # [ 0.225538] mem auto-init: stack:all(zero), heap alloc:on, heap free:off152gitlab # [ 0.236017] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1153gitlab # [ 0.250792] allocated 12582912 bytes of page_ext154gitlab # [ 0.261130] ftrace: allocating 48565 entries in 192 pages155gitlab # [ 0.261131] ftrace: allocated 192 pages with 2 groups156gitlab # [ 0.262026] Dynamic Preempt: lazy157gitlab # [ 0.262214] rcu: Preemptible hierarchical RCU implementation.158gitlab # [ 0.262215] rcu: RCU event tracing is enabled.159gitlab # [ 0.262215] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.160gitlab # [ 0.262217] Trampoline variant of Tasks RCU enabled.161gitlab # [ 0.262218] Rude variant of Tasks RCU enabled.162gitlab # [ 0.262218] Tracing variant of Tasks RCU enabled.163gitlab # [ 0.262219] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.164gitlab # [ 0.262219] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4165gitlab # [ 0.262240] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.166gitlab # [ 0.262242] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.167gitlab # [ 0.262243] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.168gitlab # [ 0.266628] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16169gitlab # [ 0.266926] rcu: srcu_init: Setting srcu_struct sizes based on contention.170gitlab # [ 0.267047] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)171gitlab # [ 0.274406] Console: colour VGA+ 80x25172gitlab # [ 0.274410] printk: legacy console [tty0] enabled173gitlab # [ 0.318287] printk: legacy console [ttyS0] enabled174gitlab # [ 0.517830] ACPI: Core revision 20250807175gitlab # [ 0.519397] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns176gitlab # [ 0.524654] APIC: Switch to symmetric I/O mode setup177gitlab # [ 0.526658] x2apic enabled178gitlab # [ 0.527864] APIC: Switched APIC routing to: physical x2apic179gitlab # [ 0.529575] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()180gitlab # [ 0.531856] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()181gitlab # [ 0.534590] kvm-guest: setup PV IPIs182gitlab # [ 0.537036] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1183gitlab # [ 0.538901] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns184gitlab # [ 0.542019] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)185gitlab # [ 0.544031] x86/cpu: User Mode Instruction Prevention (UMIP) activated186gitlab # [ 0.546142] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127187gitlab # [ 0.547017] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0188gitlab # [ 0.549021] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto189gitlab # [ 0.551017] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl190gitlab # [ 0.552017] Transient Scheduler Attacks: Mitigation: Clear CPU buffers191gitlab # [ 0.554016] Spectre V2 : Mitigation: Enhanced / Automatic IBRS192gitlab # [ 0.556017] Speculative Return Stack Overflow: Mitigation: Safe RET193gitlab # [ 0.557017] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization194gitlab # [ 0.558023] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier195gitlab # [ 0.559017] active return thunk: srso_alias_return_thunk196gitlab # [ 0.561045] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'197gitlab # [ 0.562016] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'198gitlab # [ 0.564017] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'199gitlab # [ 0.565016] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'200gitlab # [ 0.566017] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'201gitlab # [ 0.567016] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'202gitlab # [ 0.568017] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'203gitlab # [ 0.569017] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'204gitlab # [ 0.571017] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'205gitlab # [ 0.573017] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256206gitlab # [ 0.574017] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64207gitlab # [ 0.575017] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512208gitlab # [ 0.576017] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024209gitlab # [ 0.577017] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8210gitlab # [ 0.579016] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16211gitlab # [ 0.581016] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24212gitlab # [ 0.582016] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.213gitlab # [ 0.613036] Freeing SMP alternatives memory: 44K214gitlab # [ 0.614019] pid_max: default: 32768 minimum: 301215gitlab # [ 0.616058] LSM: initializing lsm=capability,landlock,yama,bpf,ima216gitlab # [ 0.618038] landlock: Up and running.217gitlab # [ 0.619017] Yama: becoming mindful.218gitlab # [ 0.620257] LSM support for eBPF active219gitlab # [ 0.622114] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)220gitlab # [ 0.623187] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)221gitlab # [ 0.626190] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)222gitlab # [ 0.627629] Performance Events: Fam17h+ core perfctr, AMD PMU driver.223gitlab # [ 0.628027] ... version: 2224gitlab # [ 0.629018] ... bit width: 48225gitlab # [ 0.630018] ... generic counters: 6226gitlab # [ 0.631018] ... generic bitmap: 000000000000003f227gitlab # [ 0.632019] ... fixed-purpose counters: 0228gitlab # [ 0.633018] ... fixed-purpose bitmap: 0000000000000000229gitlab # [ 0.634019] ... value mask: 0000ffffffffffff230gitlab # [ 0.635018] ... max period: 00007fffffffffff231gitlab # [ 0.636018] ... global_ctrl mask: 000000000000003f232gitlab # [ 0.637129] signal: max sigframe size: 3376233gitlab # [ 0.638111] rcu: Hierarchical SRCU implementation.234gitlab # [ 0.639023] rcu: Max phase no-delay instances is 400.235gitlab # [ 0.640219] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level236gitlab # [ 0.645595] smp: Bringing up secondary CPUs ...237gitlab # [ 0.646469] smpboot: x86: Booting SMP configuration:238gitlab # [ 0.647022] .... node #0, CPUs: #1 #2 #3239gitlab # [ 0.651113] smp: Brought up 1 node, 4 CPUs240gitlab # [ 0.653019] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)241gitlab # [ 0.654678] Memory: 6013144K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258292K reserved, 0K cma-reserved)242gitlab # [ 0.657018] devtmpfs: initialized243gitlab # [ 0.658256] x86/mm: Memory block size: 128MB244gitlab # [ 0.661186] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns245gitlab # [ 0.663073] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)246gitlab # [ 0.664102] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).247gitlab # [ 0.665143] pinctrl core: initialized pinctrl subsystem248gitlab # [ 0.666413] PM: RTC time: 10:22:24, date: 2026-07-05249gitlab # [ 0.670134] NET: Registered PF_NETLINK/PF_ROUTE protocol family250gitlab # [ 0.672407] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations251gitlab # [ 0.673092] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations252gitlab # [ 0.675193] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations253gitlab # [ 0.676033] audit: initializing netlink subsys (disabled)254gitlab # [ 0.677200] audit: type=2000 audit(1783246944.483:1): state=initialized audit_enabled=0 res=1255gitlab # [ 0.677335] thermal_sys: Registered thermal governor 'fair_share'256gitlab # [ 0.679019] thermal_sys: Registered thermal governor 'bang_bang'257gitlab # [ 0.680019] thermal_sys: Registered thermal governor 'step_wise'258gitlab # [ 0.681019] thermal_sys: Registered thermal governor 'user_space'259gitlab # [ 0.682019] thermal_sys: Registered thermal governor 'power_allocator'260gitlab # [ 0.683037] cpuidle: using governor menu261gitlab # [ 0.686245] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5262gitlab # [ 0.687354] PCI: Using configuration type 1 for base access263gitlab # [ 0.688019] PCI: Using configuration type 1 for extended access264gitlab # [ 0.689182] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.265gitlab # [ 0.692064] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages266gitlab # [ 0.693019] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page267gitlab # [ 0.694019] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages268gitlab # [ 0.695019] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page269gitlab # [ 0.699905] ACPI: Added _OSI(Module Device)270gitlab # [ 0.703029] ACPI: Added _OSI(Processor Device)271gitlab # [ 0.706027] ACPI: Added _OSI(Processor Aggregator Device)272gitlab # [ 0.715012] ACPI: 1 ACPI AML tables successfully acquired and loaded273gitlab # [ 0.724499] ACPI: Interpreter enabled274gitlab # [ 0.727081] ACPI: PM: (supports S0 S3 S4 S5)275gitlab # [ 0.731031] ACPI: Using IOAPIC for interrupt routing276gitlab # [ 0.735392] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug277gitlab # [ 0.742026] PCI: Using E820 reservations for host bridge windows278gitlab # [ 0.746728] ACPI: Enabled 2 GPEs in block 00 to 0F279gitlab # [ 0.762478] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])280gitlab # [ 0.766035] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]281gitlab # [ 0.773198] acpiphp: Slot [3] registered282gitlab # [ 0.775154] acpiphp: Slot [4] registered283gitlab # [ 0.778143] acpiphp: Slot [5] registered284gitlab # [ 0.781167] acpiphp: Slot [6] registered285gitlab # [ 0.783078] acpiphp: Slot [7] registered286gitlab # [ 0.784082] acpiphp: Slot [8] registered287gitlab # [ 0.785081] acpiphp: Slot [9] registered288gitlab # [ 0.787103] acpiphp: Slot [10] registered289gitlab # [ 0.788083] acpiphp: Slot [11] registered290gitlab # [ 0.789082] acpiphp: Slot [12] registered291gitlab # [ 0.791094] acpiphp: Slot [13] registered292gitlab # [ 0.792053] acpiphp: Slot [14] registered293gitlab # [ 0.793266] acpiphp: Slot [15] registered294gitlab # [ 0.797112] acpiphp: Slot [16] registered295gitlab # [ 0.800112] acpiphp: Slot [17] registered296gitlab # [ 0.803169] acpiphp: Slot [18] registered297gitlab # [ 0.806115] acpiphp: Slot [19] registered298gitlab # [ 0.810144] acpiphp: Slot [20] registered299gitlab # [ 0.813113] acpiphp: Slot [21] registered300gitlab # [ 0.816167] acpiphp: Slot [22] registered301gitlab # [ 0.819115] acpiphp: Slot [23] registered302gitlab # [ 0.823111] acpiphp: Slot [24] registered303gitlab # [ 0.826114] acpiphp: Slot [25] registered304gitlab # [ 0.829152] acpiphp: Slot [26] registered305gitlab # [ 0.832115] acpiphp: Slot [27] registered306gitlab # [ 0.836184] acpiphp: Slot [28] registered307gitlab # [ 0.839175] acpiphp: Slot [29] registered308gitlab # [ 0.842100] acpiphp: Slot [30] registered309gitlab # [ 0.845095] acpiphp: Slot [31] registered310gitlab # [ 0.847101] PCI host bridge to bus 0000:00311gitlab # [ 0.850062] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]312gitlab # [ 0.854026] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]313gitlab # [ 0.858032] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]314gitlab # [ 0.861038] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]315gitlab # [ 0.863022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]316gitlab # [ 0.869064] pci_bus 0000:00: root bus resource [bus 00-ff]317gitlab # [ 0.874352] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint318gitlab # [ 0.882874] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint319gitlab # [ 0.892508] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint320gitlab # [ 0.904214] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]321gitlab # [ 0.908193] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk322gitlab # [ 0.913026] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk323gitlab # [ 0.916026] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk324gitlab # [ 0.920025] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk325gitlab # [ 0.924690] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint326gitlab # [ 0.931111] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]327gitlab # [ 0.935439] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint328gitlab # [ 0.943951] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI329gitlab # [ 0.949064] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB330gitlab # [ 0.955230] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x180 took 10742 usecs331gitlab # [ 0.960916] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint332gitlab # [ 0.981165] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]333gitlab # [ 0.985079] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]334gitlab # [ 0.989154] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]335gitlab # [ 0.992274] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]336gitlab # [ 1.007072] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 14648 usecs337gitlab # [ 1.015138] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint338gitlab # [ 1.031064] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]339gitlab # [ 1.035064] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]340gitlab # [ 1.040187] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]341gitlab # [ 1.046065] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]342gitlab # [ 1.057321] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint343gitlab # [ 1.069061] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]344gitlab # [ 1.072053] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]345gitlab # [ 1.076090] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]346gitlab # [ 1.083599] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint347gitlab # [ 1.097063] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]348gitlab # [ 1.101191] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]349gitlab # [ 1.106175] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]350gitlab # [ 1.116241] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint351gitlab # [ 1.127058] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]352gitlab # [ 1.130053] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]353gitlab # [ 1.133074] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]354gitlab # [ 1.138072] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint355gitlab # [ 1.145337] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]356gitlab # [ 1.149054] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]357gitlab # [ 1.153139] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]358gitlab # [ 1.164287] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint359gitlab # [ 1.178064] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]360gitlab # [ 1.182054] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]361gitlab # [ 1.186132] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]362gitlab # [ 1.194240] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint363gitlab # [ 1.203064] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]364gitlab # [ 1.204038] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]365gitlab # [ 1.206100] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]366gitlab # [ 1.208034] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]367gitlab # [ 1.217027] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint368gitlab # [ 1.229095] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]369gitlab # [ 1.234167] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]370gitlab # [ 1.245189] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint371gitlab # [ 1.260094] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]372gitlab # [ 1.264063] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]373gitlab # [ 1.268131] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]374gitlab # [ 1.276788] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint375gitlab # [ 1.285063] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]376gitlab # [ 1.289063] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]377gitlab # [ 1.294161] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]378gitlab # [ 1.318372] ACPI: PCI: Interrupt link LNKA configured for IRQ 10379gitlab # [ 1.322532] ACPI: PCI: Interrupt link LNKB configured for IRQ 10380gitlab # [ 1.326443] ACPI: PCI: Interrupt link LNKC configured for IRQ 11381gitlab # [ 1.331431] ACPI: PCI: Interrupt link LNKD configured for IRQ 11382gitlab # [ 1.334238] ACPI: PCI: Interrupt link LNKS configured for IRQ 9383gitlab # [ 1.341047] iommu: Default domain type: Translated384gitlab # [ 1.342019] iommu: DMA domain TLB invalidation policy: lazy mode385gitlab # [ 1.344300] ACPI: bus type USB registered386gitlab # [ 1.346431] usbcore: registered new interface driver usbfs387gitlab # [ 1.350099] usbcore: registered new interface driver hub388gitlab # [ 1.354127] usbcore: registered new device driver usb389gitlab # [ 1.361407] NetLabel: Initializing390gitlab # [ 1.365026] NetLabel: domain hash size = 128391gitlab # [ 1.368025] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO392gitlab # [ 1.372321] NetLabel: unlabeled traffic allowed by default393gitlab # [ 1.377076] PCI: Using ACPI for IRQ routing394gitlab # [ 1.381412] pci 0000:00:02.0: vgaarb: setting as boot VGA device395gitlab # [ 1.382014] pci 0000:00:02.0: vgaarb: bridge control possible396gitlab # [ 1.382014] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none397gitlab # [ 1.395030] vgaarb: loaded398gitlab # [ 1.397459] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0399gitlab # [ 1.400023] hpet0: 3 comparators, 64-bit 100.000000 MHz counter400gitlab # [ 1.409160] clocksource: Switched to clocksource kvm-clock401gitlab # [ 1.412966] VFS: Disk quotas dquot_6.6.0402gitlab # [ 1.414263] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)403gitlab # [ 1.416716] pnp: PnP ACPI init404gitlab # [ 1.418431] pnp: PnP ACPI: found 6 devices405gitlab # [ 1.429639] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns406gitlab # [ 1.436386] clocksource: Switched to clocksource acpi_pm407gitlab # [ 1.441573] NET: Registered PF_INET protocol family408gitlab # [ 1.449769] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)409gitlab # [ 1.487873] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)410gitlab # [ 1.490529] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)411gitlab # [ 1.492964] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)412gitlab # [ 1.497967] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)413gitlab # [ 1.500380] TCP: Hash tables configured (established 65536 bind 65536)414gitlab # [ 1.502806] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)415gitlab # [ 1.505484] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)416gitlab # [ 1.507806] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)417gitlab # [ 1.510264] NET: Registered PF_UNIX/PF_LOCAL protocol family418gitlab # [ 1.512133] NET: Registered PF_XDP protocol family419gitlab # [ 1.513683] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]420gitlab # [ 1.515545] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]421gitlab # [ 1.517398] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]422gitlab # [ 1.519447] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]423gitlab # [ 1.521479] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]424gitlab # [ 1.523839] pci 0000:00:01.0: PIIX3: Enabling Passive Release425gitlab # [ 1.525678] pci 0000:00:00.0: Limiting direct PCI/PCI transfers426gitlab # [ 1.529046] ACPI: \_SB_.LNKD: Enabled at IRQ 11427gitlab # [ 1.532697] PCI: CLS 0 bytes, default 64428gitlab # [ 1.534145] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)429gitlab # [ 1.534559] Trying to unpack rootfs image as initramfs...430gitlab # [ 1.536135] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)431gitlab # [ 1.536291] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns432gitlab # [ 1.590020] Initialise system trusted keyrings433gitlab # [ 1.593504] workingset: timestamp_bits=40 max_order=21 bucket_order=0434gitlab # [ 1.617988] Key type asymmetric registered435gitlab # [ 1.619364] Asymmetric key parser 'x509' registered436gitlab # [ 1.620968] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)437gitlab # [ 1.623549] io scheduler mq-deadline registered438gitlab # [ 1.624981] io scheduler kyber registered439gitlab # [ 1.630649] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled440gitlab # [ 1.632930] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A441gitlab # [ 1.636760] Linux agpgart interface v0.103442gitlab # [ 1.638178] ACPI: bus type drm_connector registered443gitlab # [ 1.644326] usbcore: registered new interface driver usbserial_generic444gitlab # [ 1.646306] usbserial: USB Serial support registered for generic445gitlab # [ 1.648128] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.446gitlab # [ 1.648128] Please enable it if your BIOS has the CPPC option.447gitlab # [ 1.652174] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled448gitlab # [ 1.654549] drop_monitor: Initializing network drop monitor service449gitlab # [ 1.656611] NET: Registered PF_INET6 protocol family450gitlab # [ 1.659383] Segment Routing with IPv6451gitlab # [ 1.660809] In-situ OAM (IOAM) with IPv6452gitlab # [ 1.666332] IPI shorthand broadcast: enabled453gitlab # [ 1.680022] sched_clock: Marking stable (1415074480, 264079709)->(1757126817, -77972628)454gitlab # [ 1.687508] registered taskstats version 1455gitlab # [ 1.691990] Loading compiled-in X.509 certificates456gitlab # [ 1.714296] Demotion targets for Node 0: null457gitlab # [ 1.716154] Key type .fscrypt registered458gitlab # [ 1.717418] Key type fscrypt-provisioning registered459gitlab # [ 1.719208] ima: No TPM chip found, activating TPM-bypass!460gitlab # [ 1.720914] ima: Allocated hash algorithm: sha1461gitlab # [ 1.722407] ima: No architecture policies found462gitlab # [ 1.724227] PM: Magic number: 2:264:375463gitlab # [ 1.726543] RAS: Correctable Errors collector initialized.464gitlab # [ 1.731941] clk: Disabling unused clocks465gitlab # [ 1.733256] PM: genpd: Disabling unused power domains466gitlab # [ 1.861818] Freeing initrd memory: 27584K467gitlab # [ 1.866624] Freeing unused decrypted memory: 2028K468gitlab # [ 1.870712] Freeing unused kernel image (initmem) memory: 3644K469gitlab # [ 1.875592] Write protecting the kernel read-only data: 32768k470gitlab # [ 1.883225] Freeing unused kernel image (text/rodata gap) memory: 1332K471gitlab # [ 1.889675] Freeing unused kernel image (rodata/data gap) memory: 816K472gitlab # [ 1.955234] x86/mm: Checked W+X mappings: passed, no W+X pages found.473gitlab # [ 1.957306] Run /init as init process474gitlab # [ 1.971298] systemd[1]: Inserted module 'autofs4'475gitlab # [ 1.999332] fuse: init (API version 7.45)476gitlab # [ 2.009547] ACPI: \_SB_.LNKC: Enabled at IRQ 10477gitlab # [ 2.020337] ACPI: \_SB_.LNKA: Enabled at IRQ 10478gitlab # [ 2.037391] ACPI: \_SB_.LNKB: Enabled at IRQ 11479gitlab # [ 2.092818] systemd[1]: Successfully made /usr/ read-only.480gitlab # [ 2.432919] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)481gitlab # [ 2.454613] systemd[1]: Detected virtualization kvm.482gitlab # [ 2.458618] systemd[1]: Detected architecture x86-64.483gitlab # [ 2.462660] systemd[1]: Running in initrd.484gitlab # [ 2.466982] systemd[1]: Initializing machine ID from random generator.485gitlab # [ 2.472370] systemd[1]: Hostname set to <gitlab>.486gitlab # [ 2.551411] systemd[1]: Queued start job for default target Initrd Default Target.487gitlab # [ 2.721561] systemd[1]: Created slice Slice /system/modprobe.488gitlab # [ 2.723727] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.489gitlab # [ 2.729467] systemd[1]: Expecting device /dev/disk/by-label/nix-store...490gitlab # [ 2.734669] systemd[1]: Expecting device /dev/disk/by-label/nixos...491gitlab # [ 2.739752] systemd[1]: Reached target Path Units.492gitlab # [ 2.743788] systemd[1]: Reached target Slice Units.493gitlab # [ 2.747822] systemd[1]: Reached target Swaps.494gitlab # [ 2.751040] systemd[1]: Reached target Timer Units.495gitlab # [ 2.754532] systemd[1]: Listening on D-Bus System Message Bus Socket.496gitlab # [ 2.758920] systemd[1]: Listening on Journal Socket (/dev/log).497gitlab # [ 2.762937] systemd[1]: Listening on Journal Sockets.498gitlab # [ 2.766527] systemd[1]: Listening on udev Control Socket.499gitlab # [ 2.769243] systemd[1]: Listening on udev Kernel Socket.500gitlab # [ 2.771001] systemd[1]: Reached target Socket Units.501gitlab # [ 2.776924] systemd[1]: Starting Create List of Static Device Nodes...502gitlab # [ 2.782919] systemd[1]: Starting Load Kernel Module 9pnet_virtio...503gitlab # [ 2.802628] systemd[1]: Starting Load Kernel Module configfs...504gitlab # [ 2.808380] systemd[1]: Starting Journal Service...505gitlab # [ 2.816046] systemd[1]: Starting Load Kernel Modules...506gitlab # [ 2.817839] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki507gitlab # [ 2.827749] systemd[1]: Starting Coldplug All udev Devices...508gitlab # [ 2.845723] systemd-journald[152]: Collecting audit messages is disabled.509gitlab # [ 2.856680] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.510gitlab # [ 2.859442] netfs: FS-Cache loaded511gitlab # [ 2.860633] systemd[1]: Finished Create List of Static Device Nodes.512gitlab # [ 2.863254] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev513gitlab # [ 2.864337] systemd[1]: modprobe@configfs.service: Deactivated successfully.514gitlab # [ 2.870428] systemd[1]: Finished Load Kernel Module configfs.515gitlab # [ 2.875329] 9pnet: Installing 9P2000 support516gitlab # [ 2.877432] systemd[1]: Finished Load Kernel Modules.517gitlab # [ 2.879890] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config518gitlab # [ 2.886410] systemd[1]: Starting Apply Kernel Variables...519gitlab # [ 2.897434] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...520gitlab # [ 2.903899] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.521gitlab # [ 2.915702] systemd[1]: Finished Load Kernel Module 9pnet_virtio.522gitlab # [ 2.656242] systemd-modules-load[155]: Using 4 probe threads523gitlab # [ 2.921613] systemd[1]: Started Journal Service.524gitlab # [ 2.658473] systemd-modules-load[155]: Inserted module 'virtio_balloon'525gitlab # [ 2.660784] systemd-modules-load[155]: Inserted module 'dm_mod'526gitlab # [ 2.662495] systemd-modules-load[155]: Inserted module 'virtio_gpu'527gitlab # [ 2.664289] systemd[1]: Finished Apply Kernel Variables.528gitlab # [ 2.665982] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.529gitlab # [ 2.672943] systemd[1]: Starting Create Static Device Nodes in /dev...530gitlab # [ 2.689486] systemd[1]: Finished Create Static Device Nodes in /dev.531gitlab # [ 2.691094] systemd[1]: Reached target Preparation for Local File Systems.532gitlab # [ 2.692710] systemd[1]: Reached target Local File Systems.533gitlab # [ 2.694307] systemd[1]: Starting Create System Files and Directories...534gitlab # [ 2.696629] systemd[1]: Starting Rule-based Manager for Device Events and Files...535gitlab # [ 2.717746] systemd[1]: Finished Create System Files and Directories.536gitlab # [ 2.724307] systemd-udevd[193]: Using default interface naming scheme 'v260'.537gitlab # [ 2.740605] systemd[1]: Started Rule-based Manager for Device Events and Files.538gitlab # [ 2.767063] systemd[1]: Finished Coldplug All udev Devices.539gitlab # [ 2.769726] systemd[1]: Reached target System Initialization.540gitlab # [ 2.772428] systemd[1]: Reached target Basic System.541gitlab # [ 3.165096] virtio_blk virtio4: 4/0/0 default/read/poll queues542gitlab # [ 3.174575] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)543gitlab # [ 3.175294] uhci_hcd 0000:00:01.2: UHCI Host Controller544gitlab # [ 3.179580] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1545gitlab # [ 3.181452] uhci_hcd 0000:00:01.2: detected 2 ports546gitlab # [ 3.184251] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140547gitlab # [ 3.185936] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18548gitlab # [ 3.190173] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1549gitlab # [ 3.191927] usb usb1: Product: UHCI Host Controller550gitlab # [ 3.192314] virtio_blk virtio5: 4/0/0 default/read/poll queues551gitlab # [ 3.193358] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd552gitlab # [ 3.194767] SCSI subsystem initialized553gitlab # [ 3.195114] usb usb1: SerialNumber: 0000:00:01.2554gitlab # [ 3.196838] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12555gitlab # [ 3.202051] serio: i8042 KBD port at 0x60,0x64 irq 1556gitlab # [ 3.203980] serio: i8042 AUX port at 0x60,0x64 irq 12557gitlab # [ 3.204324] virtio_blk virtio5: [vdb] 11450592 512-byte logical blocks (5.86 GB/5.46 GiB)558gitlab # [ 3.205196] hub 1-0:1.0: USB hub found559gitlab # [ 3.208895] hub 1-0:1.0: 2 ports detected560gitlab # [ 3.228034] scsi host0: ata_piix561gitlab # [ 3.230023] scsi host1: ata_piix562gitlab # [ 3.265194] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0563gitlab # [ 3.003577] (udev-worker)[220]: eth1:[ 3.267959] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0564gitlab # Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.565gitlab # [ 3.009179] (udev-worker)[220]: Network interface NamePolicy= disabled on kernel command line.566gitlab # [ 3.011605] (udev-worker)[203]: Network interface NamePolicy= disabled on kernel command line.567gitlab # [ 3.023885] systemd[1]: Starting Virtual Console Setup...568gitlab # [ 3.030055] systemd[1]: Found device /dev/disk/by-label/nixos.569gitlab # [ 3.032482] systemd[1]: Reached target Initrd Root Device.570gitlab # [ 3.034921] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...571gitlab # [ 3.039043] systemd[1]: Found device /dev/disk/by-label/nix-store.572gitlab # [ 3.047857] systemd-vconsole-setup[234]: Configuration of first virtual console was skipped, ignoring remaining ones.573gitlab # [ 3.052223] systemd[1]: Finished Virtual Console Setup.574gitlab # [ 3.064245] systemd-fsck[242]: nixos: clean, 12/65536 files, 13019/262144 blocks575gitlab # [ 3.068731] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.576gitlab # [ 3.428541] ata2: found unknown device (class 0)577gitlab # [ 3.429190] usb 1-1: new full-speed USB device number 2 using uhci_hcd578gitlab # [ 3.432275] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100579gitlab # [ 3.442685] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5580gitlab # [ 3.484045] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0581gitlab # [ 3.527348] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray582gitlab # [ 3.557509] cdrom: Uniform CD-ROM driver Revision: 3.20583gitlab # [ 3.604345] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00584gitlab # [ 3.608325] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10585gitlab # [ 3.611905] usb 1-1: Product: QEMU USB Tablet586gitlab # [ 3.614217] usb 1-1: Manufacturer: QEMU587gitlab # [ 3.616253] usb 1-1: SerialNumber: 28754-0000:00:01.2-1588gitlab # [ 3.654572] hid: raw HID events driver (C) Jiri Kosina589gitlab # [ 3.671310] usbcore: registered new interface driver usbhid590gitlab # [ 3.675185] usbhid: USB HID core driver591gitlab # [ 3.690424] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2592gitlab # [ 3.697643] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0593gitlab # [ 3.590511] systemd[1]: Mounting /sysroot...594gitlab # [ 4.035378] EXT4-fs (vda): mounted filesystem c8d7737a-61ed-43cc-9422-826f5aa5bee5 r/w with ordered data mode. Quota mode: none.595gitlab # [ 3.775805] systemd[1]: Mounted /sysroot.596gitlab # [ 3.777657] systemd[1]: Reached target Initrd Root File System.597gitlab # [ 3.784984] systemd[1]: Mounting /sysroot/nix/.ro-store...598gitlab # [ 3.798890] systemd[1]: Mounting /sysroot/nix/.rw-store...599gitlab # [ 3.803155] systemd[1]: Mounting /sysroot/run...600gitlab # [ 3.810417] systemd[1]: Mounting /sysroot/tmp/shared...601gitlab # [ 4.076913] erofs (device vdb): mounted with root inode @ nid 128.602gitlab # [ 3.815339] systemd[1]: Mounting /sysroot/tmp/xchg...603gitlab # [ 3.821212] systemd[1]: Starting Mountpoints Configured in the Real Root...604gitlab # [ 4.089138] 9p: Installing v9fs 9p2000 file system support605gitlab # [ 3.827968] systemd-sysroot-fstab-check[271]: /sysroot should be mounted in the initrd, will request daemon-reload.606gitlab # [ 3.833795] systemd[1]: Mounted /sysroot/nix/.ro-store.607gitlab # [ 3.835212] systemd[1]: Mounted /sysroot/nix/.rw-store.608gitlab # [ 3.836528] systemd[1]: Mounted /sysroot/run.609gitlab # [ 3.837680] systemd[1]: Mounted /sysroot/tmp/shared.610gitlab # [ 3.838932] systemd[1]: Mounted /sysroot/tmp/xchg.611gitlab # [ 3.842139] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.843637] systemd[1]: Reload requested from client PID 271 ('systemd-sysroot') (unit initrd-parse-etc.service)...613gitlab # [ 3.846291] systemd[1]: Reloading...614gitlab # [ 4.050867] systemd[1]: Reloading finished in 207 ms.615gitlab # [ 4.102914] systemd-sysroot-fstab-check[271]: Requesting initrd-fs.target/start/replace...616gitlab # [ 4.242339] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.617gitlab # [ 4.250547] systemd[1]: Finished rw-sysroot-nix-store.service.618gitlab # [ 4.264589] systemd-sysroot-fstab-check[271]: Requesting swap.target/start/replace...619gitlab # [ 4.273316] systemd[1]: initrd-parse-etc.service: Deactivated successfully.620gitlab # [ 4.281876] systemd[1]: Finished Mountpoints Configured in the Real Root.621gitlab # [ 4.285922] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.622gitlab # [ 4.300860] systemd[1]: Starting rw-sysroot-nix-store.service...623gitlab # [ 4.331794] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.624gitlab # [ 4.340778] systemd[1]: Finished rw-sysroot-nix-store.service.625gitlab # [ 4.592938] systemd[1]: Mounting /sysroot/nix/store...626gitlab # [ 4.651679] systemd[1]: Mounted /sysroot/nix/store.627gitlab # [ 4.655576] systemd[1]: Reached target Initrd File Systems.628gitlab # [ 4.663758] systemd[1]: Starting Find NixOS closure...629gitlab # [ 4.669781] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...630gitlab # [ 4.707200] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.631gitlab # [ 4.712351] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.632gitlab # [ 4.731802] systemd[1]: Finished Find NixOS closure.633gitlab # [ 4.734581] systemd[1]: Reached target Initrd Default Target.634gitlab # [ 4.738808] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...635gitlab # [ 4.759490] systemd[1]: Stopped target Initrd Default Target.636gitlab # [ 4.763191] systemd[1]: Stopped target Basic System.637gitlab # [ 4.766433] systemd[1]: Stopped target Initrd Root Device.638gitlab # [ 4.769844] systemd[1]: Stopped target Path Units.639gitlab # [ 4.772835] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.640gitlab # [ 4.777388] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.641gitlab # [ 4.782216] systemd[1]: Stopped target Slice Units.642gitlab # [ 4.784732] systemd[1]: Stopped target Socket Units.643gitlab # [ 4.787251] systemd[1]: Stopped target System Initialization.644gitlab # [ 4.790139] systemd[1]: Stopped target Swaps.645gitlab # [ 4.792563] systemd[1]: Stopped target Timer Units.646gitlab # [ 4.795461] systemd[1]: dbus.socket: Deactivated successfully.647gitlab # [ 4.798193] systemd[1]: Closed D-Bus System Message Bus Socket.648gitlab # [ 4.800516] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.649gitlab # [ 4.802889] systemd[1]: Stopped Find NixOS closure.650gitlab # [ 4.804418] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio651gitlab # [ 4.806993] systemd[1]: Starting rw-sysroot-nix-store.service...652gitlab # [ 4.808470] systemd[1]: systemd-sysctl.service: Deactivated successfully.653gitlab # [ 4.810099] systemd[1]: Stopped Apply Kernel Variables.654gitlab # [ 4.811386] systemd[1]: systemd-modules-load.service: Deactivated successfully.655gitlab # [ 4.813146] systemd[1]: Stopped Load Kernel Modules.656gitlab # [ 4.814405] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.657gitlab # [ 4.816302] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.658gitlab # [ 4.818227] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.659gitlab # [ 4.820920] systemd[1]: Stopped Create System Files and Directories.660gitlab # [ 4.822472] systemd[1]: Stopped target Local File Systems.661gitlab # [ 4.823880] systemd[1]: Stopped target Preparation for Local File Systems.662gitlab # [ 4.825479] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.663gitlab # [ 4.827232] systemd[1]: Stopped Coldplug All udev Devices.664gitlab # [ 4.828660] systemd[1]: Stopping Rule-based Manager for Device Events and Files...665gitlab # [ 4.830523] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.666gitlab # [ 4.832379] systemd[1]: Stopped Virtual Console Setup.667gitlab # [ 4.833684] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.668gitlab # [ 4.835563] systemd[1]: Finished rw-sysroot-nix-store.service.669gitlab # [ 4.839141] systemd[1]: systemd-udevd.service: Deactivated successfully.670gitlab # [ 4.842613] systemd[1]: Stopped Rule-based Manager for Device Events and Files.671gitlab # [ 4.844467] systemd[1]: systemd-udevd.service: Consumed 1.042s CPU time over 2.142s wall clock time, 25.2M memory peak.672gitlab # [ 4.852732] systemd[1]: initrd-cleanup.service: Deactivated successfully.673gitlab # [ 4.856053] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.674gitlab # [ 4.862510] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.675gitlab # [ 4.864340] systemd[1]: Closed udev Control Socket.676gitlab # [ 4.869213] systemd[1]: Starting Cleanup udev Database...677gitlab # [ 4.870605] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.678gitlab # [ 4.872421] systemd[1]: Stopped Create Static Device Nodes in /dev.679gitlab # [ 4.875217] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.680gitlab # [ 4.877137] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.681gitlab # [ 4.879938] systemd[1]: kmod-static-nodes.service: Deactivated successfully.682gitlab # [ 4.881641] systemd[1]: Stopped Create List of Static Device Nodes.683gitlab # [ 4.888761] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.684gitlab # [ 4.892607] systemd[1]: Finished Cleanup udev Database.685gitlab # [ 4.896673] systemd[1]: Reached target Switch Root.686gitlab # [ 4.899462] systemd[1]: Starting NixOS Activation...687gitlab # [ 5.020729] initrd-nixos-activation-start[569]: booting system configuration /nix/store/34xs09d1jkfapqaxyzjrxrgwkmr28bas-nixos-system-gitlab-test688gitlab # [ 5.061288] initrd-nixos-activation-start[569]: running activation script...689gitlab # [ 5.296720] initrd-nixos-activation-start[592]: setting up /etc...690gitlab # [ 5.430818] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.691gitlab # [ 5.435801] systemd[1]: Finished NixOS Activation.692gitlab # [ 5.448268] systemd[1]: Starting Switch Root...693gitlab # [ 5.490185] systemd[1]: Switching root.694gitlab # [ 6.136263] systemd-journald[152]: Received SIGTERM from PID 1 (systemd).695gitlab # [ 6.269799] NET: Registered PF_VSOCK protocol family696gitlab # [ 6.641012] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)697gitlab # [ 6.657710] systemd[1]: Detected virtualization kvm.698gitlab # [ 6.660957] systemd[1]: Detected architecture x86-64.699gitlab # [ 6.664391] systemd[1]: Detected first boot.700gitlab # [ 6.668576] systemd[1]: Initializing machine ID from random generator.701gitlab # [ 6.810381] systemd[1]: bpf-restrict-fs: LSM BPF program attached702gitlab # [ 6.878245] systemd[1]: Applying preset policy.703gitlab # [ 7.053256] systemd[1]: Populated /etc with preset unit settings.704gitlab # [ 7.172168] systemd[1]: initrd-switch-root.service: Deactivated successfully.705gitlab # [ 7.174604] systemd[1]: Stopped initrd-switch-root.service.706gitlab # [ 7.177895] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.707gitlab # [ 7.181582] systemd[1]: Created slice Slice /system/getty.708gitlab # [ 7.183755] systemd[1]: Created slice GitLab DevOps Platform Slice.709gitlab # [ 7.186016] systemd[1]: Created slice User and Session Slice.710gitlab # [ 7.187645] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.711gitlab # [ 7.189723] systemd[1]: Started Forward Password Requests to Wall Directory Watch.712gitlab # [ 7.191615] systemd[1]: Expecting device /dev/hvc0...713gitlab # [ 7.192941] systemd[1]: Expecting device /dev/ttyS0...714gitlab # [ 7.194341] systemd[1]: Reached target Local Encrypted Volumes.715gitlab # [ 7.195868] systemd[1]: Stopped target initrd-fs.target.716gitlab # [ 7.197270] systemd[1]: Stopped target initrd-root-fs.target.717gitlab # [ 7.198749] systemd[1]: Stopped target initrd-switch-root.target.718gitlab # [ 7.200371] systemd[1]: Reached target Virtual Machines and Containers.719gitlab # [ 7.202100] systemd[1]: Reached target Path Units.720gitlab # [ 7.203486] systemd[1]: Reached target Remote File Systems.721gitlab # [ 7.204928] systemd[1]: Reached target Slice Units.722gitlab # [ 7.206287] systemd[1]: Reached target Swaps.723gitlab # [ 7.208616] systemd[1]: Listening on Process Core Dump Socket.724gitlab # [ 7.210784] systemd[1]: Listening on Credential Encryption/Decryption.725gitlab # [ 7.214375] systemd[1]: Starting Journal Log Access Socket...726gitlab # [ 7.216690] systemd[1]: Listening on Journal Audit Socket.727gitlab # [ 7.218431] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.728gitlab # [ 7.220349] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki729gitlab # [ 7.222789] systemd[1]: Listening on udev Control Socket.730gitlab # [ 7.226727] systemd[1]: Mounting Huge Pages File System...731gitlab # [ 7.229720] systemd[1]: Mounting POSIX Message Queue File System...732gitlab # [ 7.232931] systemd[1]: Mounting Kernel Debug File System...733gitlab # [ 7.237555] systemd[1]: Mounting Kernel Trace File System...734gitlab # [ 7.247601] systemd[1]: Starting Create List of Static Device Nodes...735gitlab # [ 7.249311] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio736gitlab # [ 7.253734] systemd[1]: Starting Load Kernel Module configfs...737gitlab # [ 7.255527] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm738gitlab # [ 7.255629] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore739gitlab # [ 7.255713] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse740gitlab # [ 7.260427] systemd[1]: Mounting FUSE Control File System...741gitlab # [ 7.280440] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67742gitlab # [ 7.302978] systemd[1]: Starting Journal Service...743gitlab # [ 7.310252] systemd[1]: Starting Load Kernel Modules...744gitlab # [ 7.339529] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...745gitlab # [ 7.343140] systemd[1]: Starting Remount Root and Kernel File Systems...746gitlab # [ 7.344818] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki747gitlab # [ 7.351195] systemd[1]: Starting Coldplug All udev Devices...748gitlab # [ 7.356946] systemd[1]: Listening on Journal Log Access Socket.749gitlab # [ 7.359313] systemd[1]: Mounted Huge Pages File System.750gitlab # [ 7.359567] systemd[1]: Mounted POSIX Message Queue File System.751gitlab # [ 7.362163] systemd[1]: Mounted Kernel Debug File System.752gitlab # [ 7.362364] loop: module loaded753gitlab # [ 7.362391] systemd[1]: Mounted Kernel Trace File System.754gitlab # [ 7.365155] systemd-journald[790]: Collecting audit messages is enabled.755gitlab # [ 7.366578] systemd[1]: Finished Create List of Static Device Nodes.756gitlab # [ 7.372563] systemd[1]: modprobe@configfs.service: Deactivated successfully.757gitlab # [ 7.375172] systemd[1]: Finished Load Kernel Module configfs.758gitlab # [ 7.377706] systemd[1]: Mounted FUSE Control File System.759gitlab # [ 7.122170] systemd[1]: Queued start job for default target Multi-User System.760gitlab # [ 7.388960] systemd[1]: Mounting Kernel Configuration File System...761gitlab # [ 7.127521] systemd[1]: Unnecessary job was removed for /dev/vdb.762gitlab # [ 7.129779] systemd[1]: systemd-journald.service: Deactivated successfully.763gitlab # [ 7.395195] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...764gitlab # [ 7.132148] systemd-modules-load[791]: Using 3 probe threads[ 7.397849] systemd[1]: Started Journal Service.765gitlab # [ 7.400142] EXT4-fs (vda): re-mounted c8d7737a-61ed-43cc-9422-826f5aa5bee5.766gitlab # 767gitlab # [ 7.138456] systemd-modules-load[791]: Inserted module 'loop'768gitlab # [ 7.141140] systemd-modules-load[791]: Inserted module 'tls'769gitlab # [ 7.144493] systemd[1]: Finished Load Kernel Modules.770gitlab # [ 7.146848] systemd[1]: Finished Remount Root and Kernel File Systems.771gitlab # [ 7.151537] systemd-oomd[793]: No swap; memory pressure usage will be degraded772gitlab # [ 7.154712] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.773gitlab # [ 7.156805] systemd[1]: Starting Firewall...774gitlab # [ 7.159288] systemd[1]: Starting Flush Journal to Persistent Storage...775gitlab # [ 7.161186] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore776gitlab # [ 7.163965] systemd[1]: Starting Load/Save OS Random Seed...777gitlab # [ 7.165578] systemd[1]: Starting Apply Kernel Variables...778gitlab # [ 7.167103] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki779gitlab # [ 7.169363] systemd[1]: Mounted Kernel Configuration File System.780gitlab # [ 7.173254] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.781gitlab # [ 7.442999] systemd-journald[790]: Received client request to flush runtime journal.782gitlab # [ 7.241586] systemd[1]: Starting Create Static Device Nodes in /dev...783gitlab # [ 7.243792] systemd[1]: Finished Load/Save OS Random Seed.784gitlab # [ 7.245457] systemd[1]: Reached target First Boot Complete.785gitlab # [ 7.247140] systemd[1]: Finished Apply Kernel Variables.786gitlab # [ 7.248521] systemd[1]: Finished Create Static Device Nodes in /dev.787gitlab # [ 7.250160] systemd[1]: Reached target Preparation for Local File Systems.788gitlab # [ 7.251832] systemd[1]: Starting Rule-based Manager for Device Events and Files...789gitlab # [ 7.253613] systemd[1]: Finished Flush Journal to Persistent Storage.790gitlab # [ 7.264649] systemd-udevd[824]: Using default interface naming scheme 'v260'.791gitlab # [ 7.308318] systemd[1]: Finished Coldplug All udev Devices.792gitlab # [ 7.310434] systemd[1]: Started Rule-based Manager for Device Events and Files.793gitlab # [ 7.432970] systemd[1]: Finished Firewall.794gitlab # [ 7.444558] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse795gitlab # [ 7.469254] (udev-worker)[925]: Network interface NamePolicy= disabled on kernel command line.796gitlab # [ 7.471982] (udev-worker)[936]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.797gitlab # [ 7.476365] (udev-worker)[936]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 7.478828] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.799gitlab # [ 7.491854] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.800gitlab # [ 7.512955] systemd[1]: Condition check resulted in Virtio network device being skipped.801gitlab # [ 7.515143] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore802gitlab # [ 7.517817] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67803gitlab # [ 7.522345] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore804gitlab # [ 7.525300] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki805gitlab # [ 7.527589] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki806gitlab # [ 7.840825] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console807gitlab # [ 7.841648] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3808gitlab # [ 7.845383] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0809gitlab # [ 7.846549] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD810gitlab # [ 7.858691] ACPI: button: Power Button [PWRF]811gitlab # [ 7.866666] Console: switching to colour dummy device 80x25812gitlab # [ 7.871674] [drm] Found bochs VGA, ID 0xb0c5.813gitlab # [ 7.871683] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.814gitlab # [ 7.874159] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic815gitlab # [ 7.875535] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0816gitlab # [ 7.876804] parport_pc 00:03: reported by Plug and Play ACPI817gitlab # [ 7.879869] rtc_cmos 00:05: RTC can wake from S4818gitlab # [ 7.881506] Floppy drive(s): fd0 is 2.88M AMI BIOS819gitlab # [ 7.883526] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]820gitlab # [ 7.884787] rtc_cmos 00:05: registered as rtc0821gitlab # [ 7.885881] rtc_cmos 00:05: setting system clock to 2026-07-05T10:22:31 UTC (1783246951)822gitlab # [ 7.887163] mousedev: PS/2 mouse device common for all mice823gitlab # [ 7.887643] systemd-journald[790]: Time jumped backwards, rotating.824gitlab # [ 7.889469] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs825gitlab # [ 7.892607] fbcon: bochs-drmdrmfb (fb0) is primary device826gitlab # [ 7.896946] FDC 0 is a S82078B827gitlab # [ 7.899731] Console: switching to colour frame buffer device 160x50828gitlab # [ 7.903540] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device829gitlab # [ 7.934130] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5830gitlab # [ 7.937513] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4831gitlab # [ 7.957428] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6832gitlab # [ 7.724297] systemd[1]: Starting Virtual Console Setup...833gitlab # [ 7.756112] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.834gitlab # [ 7.758425] systemd[1]: Stopped Virtual Console Setup.835gitlab # [ 7.769112] systemd[1]: Starting Virtual Console Setup...836gitlab # [ 8.058501] ppdev: user-space parallel port driver837gitlab # [ 7.795634] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.838gitlab # [ 7.797691] systemd[1]: Stopped Virtual Console Setup.839gitlab # [ 7.806825] systemd[1]: Starting Virtual Console Setup...840gitlab # [ 8.088921] kvm_amd: TSC scaling supported841gitlab # [ 8.089994] kvm_amd: Nested Virtualization enabled842gitlab # [ 8.091268] kvm_amd: Nested Paging enabled843gitlab # [ 8.092376] kvm_amd: LBR virtualization supported844gitlab # [ 8.093597] kvm_amd: Virtual VMLOAD VMSAVE supported845gitlab # [ 8.094875] kvm_amd: Virtual GIF supported846gitlab # [ 8.095796] kvm_amd: Virtual NMI enabled847gitlab # [ 8.147592] EDAC MC: Ver: 3.0.0848gitlab # [ 7.888320] systemd-vconsole-setup[965]: Configuration of first virtual console was skipped, ignoring remaining ones.849gitlab # [ 7.892289] systemd[1]: Finished Virtual Console Setup.850gitlab # [ 7.909878] systemd[1]: Mounting /run/wrappers...851gitlab # [ 7.930710] systemd[1]: Mounted /run/wrappers.852gitlab # [ 7.932607] systemd[1]: Reached target Local File Systems.853gitlab # [ 7.934688] systemd[1]: Listening on Boot Loader Control Service Socket.854gitlab # [ 7.937562] systemd[1]: Starting register-nix-paths.service...855gitlab # [ 7.940212] systemd[1]: Starting Create SUID/SGID Wrappers...856gitlab # [ 7.941852] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.857gitlab # [ 7.952855] systemd[1]: Starting Save Transient machine-id to Disk...858gitlab # [ 7.955765] systemd[1]: Starting Create System Files and Directories...859gitlab # [ 7.987803] systemd[1]: Finished Create System Files and Directories.860gitlab # [ 7.991417] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.861gitlab # [ 7.993749] systemd[1]: Finished Save Transient machine-id to Disk.862gitlab # [ 7.997939] systemd[1]: Starting Rebuild Journal Catalog...863gitlab # [ 8.001502] systemd[1]: Starting Record System Boot/Shutdown in UTMP...864gitlab # [ 8.017346] systemd[1]: Finished Record System Boot/Shutdown in UTMP.865gitlab # [ 8.031525] systemd[1]: Finished Rebuild Journal Catalog.866gitlab # [ 8.033873] systemd[1]: Starting Update is Completed...867gitlab # [ 8.051312] systemd[1]: Finished Update is Completed.868gitlab # [ 8.090241] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.869gitlab # [ 8.092246] systemd[1]: Finished Create SUID/SGID Wrappers.870gitlab # [ 8.093712] systemd[1]: suid-sgid-wrappers.service: Consumed 114ms CPU time over 149ms wall clock time, 5M memory peak, 1.1M read from disk.871gitlab # [ 8.357145] systemd[1]: Finished register-nix-paths.service.872gitlab # [ 8.361804] systemd[1]: Reached target System Initialization.873gitlab # [ 8.366299] systemd[1]: Started Discard unused filesystem blocks once a week.874gitlab # [ 8.370643] systemd[1]: Started logrotate.timer.875gitlab # [ 8.373839] systemd[1]: Started Daily Cleanup of Temporary Directories.876gitlab # [ 8.377896] systemd[1]: Reached target Timer Units.877gitlab # [ 8.381212] systemd[1]: Listening on D-Bus System Message Bus Socket.878gitlab # [ 8.385485] systemd[1]: Listening on Nix Daemon Socket.879gitlab # [ 8.400132] systemd[1]: Starting nixbot CI service socket...880gitlab # [ 8.403754] systemd[1]: Listening on Hostname Service Socket.881gitlab # [ 8.407499] systemd[1]: Starting D-Bus System Message Bus...882gitlab # [ 8.411370] systemd[1]: Listening on nixbot CI service socket.883gitlab # [ 8.414922] systemd[1]: Reached target Socket Units.884gitlab # [ 8.445808] dbus-broker-launch[1055]: Looking up NSS user entry for 'systemd-timesync'...885gitlab # [ 8.451721] dbus-broker-launch[1055]: NSS returned no entry for 'systemd-timesync'886gitlab # [ 8.456300] dbus-broker-launch[1055]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"887gitlab # [ 8.465668] systemd[1]: Started D-Bus System Message Bus.888gitlab # [ 8.481414] dbus-broker-launch[1055]: Ready889gitlab # [ 8.484758] systemd[1]: Reached target Basic System.890gitlab # [ 8.488292] systemd[1]: Started backdoor.service.891gitlab # [ 8.491495] systemd[1]: Starting gitlab-config.service...892gitlab # [ 8.497686] systemd[1]: Starting Import lastlog data into lastlog2 database...893gitlab # [ 8.502516] systemd[1]: Starting Logrotate configuration check...894gitlab # [ 8.505454] systemd[1]: Starting Name Service Cache Daemon (nsncd)...895gitlab # [ 8.508684] systemd[1]: Starting Post-Boot Actions...896gitlab # [ 8.519294] systemd[1]: Started Reset console on configuration changes.897gitlab # [ 8.524270] systemd[1]: Starting resolvconf update...898gitlab # [ 8.529285] logrotate[1060]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.899gitlab # [ 8.532996] logrotate[1060]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf900gitlab # [ 8.535357] logrotate[1060]: note: 'monthly' overrides previously specified 'weekly'901gitlab # [ 8.537184] logrotate[1060]: Reading state from file: /var/lib/logrotate.status902gitlab # connecting to host...903gitlab # [ 8.539128] logrotate[1060]: state file /var/lib/logrotate.status does not exist904gitlab # [ 8.540867] logrotate[1060]: Allocating hash table for state file, size 64 entries905gitlab # [ 8.541870] logrotate[1060]: Handling 4 logs906gitlab: Guest shell says: b'Spawning backdoor root shell...\n'907gitlab: connected to guest root shell908gitlab: (connecting took 9.40 seconds)909gitlab: (finished: waiting for the VM to finish booting, in 9.40 seconds)910gitlab # [ 8.542918] logrotate[1060]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed911gitlab # [ 8.544145] logrotate[1060]: considering log /var/log/btmp912gitlab # [ 8.545092] logrotate[1060]: Creating new state913gitlab # [ 8.545945] logrotate[1060]: Now: 2026-07-05 10:22914gitlab # [ 8.546911] logrotate[1060]: Last rotated at 2026-07-05 10:00915gitlab # [ 8.548218] logrotate[1060]: log does not need rotating (log has already been rotated)916gitlab # [ 8.549123] logrotate[1060]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed917gitlab # [ 8.549738] logrotate[1060]: considering log /var/log/wtmp918gitlab # [ 8.550893] logrotate[1060]: Creating new state919gitlab # [ 8.552194] logrotate[1060]: Now: 2026-07-05 10:22920gitlab # [ 8.553112] logrotate[1060]: Last rotated at 2026-07-05 10:00921gitlab # [ 8.553934] logrotate[1060]: log does not need rotating (log has already been rotated)922gitlab # [ 8.555338] logrotate[1060]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed923gitlab # [ 8.556204] logrotate[1060]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1060)924gitlab # [ 8.556763] logrotate[1060]: considering log /var/gitlab/state/log/*.log925gitlab # [ 8.557881] logrotate[1060]: log /var/gitlab/state/log/*.log does not exist -- skipping926gitlab # [ 8.559690] logrotate[1060]: Creating new state927gitlab # [ 8.560851] logrotate[1060]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1060)928gitlab # [ 8.562093] logrotate[1060]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed929gitlab # [ 8.562907] logrotate[1060]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1060)930gitlab # [ 8.564896] logrotate[1060]: considering log /var/log/nginx/*.log931gitlab # [ 8.566128] logrotate[1060]: log /var/log/nginx/*.log does not exist -- skipping932gitlab # [ 8.567323] logrotate[1060]: Creating new state933gitlab # [ 8.568127] logrotate[1060]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1060)934gitlab # [ 8.569159] nsncd[1061]: Jul 05 10:22:32.433 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"935gitlab # [ 8.588369] systemd[1]: Started Name Service Cache Daemon (nsncd).936gitlab # [ 8.592592] systemd[1]: Finished Logrotate configuration check.937gitlab # [ 8.596171] systemd[1]: Finished Post-Boot Actions.938gitlab # [ 8.602197] systemd[1]: Finished Import lastlog data into lastlog2 database.939gitlab # [ 8.616923] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...940gitlab # [ 8.620134] systemd[1]: nscd.service: Deactivated successfully.941gitlab # [ 8.621240] systemd[1]: Stopped Name Service Cache Daemon (nsncd).942gitlab # [ 8.621800] systemd[1]: nscd.service: Consumed 26ms CPU time over 121ms wall clock time, 8M memory peak, 5.7M read from disk.943gitlab # [ 8.625930] systemd[1]: Starting Name Service Cache Daemon (nsncd)...944gitlab # [ 8.645416] systemd[1]: Started Name Service Cache Daemon (nsncd).945gitlab # [ 8.647193] systemd[1]: Reached target Host and Network Name Lookups.946gitlab # [ 8.649357] nsncd[1135]: Jul 05 10:22:32.525 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"947gitlab # [ 8.650811] systemd[1]: Reached target User and Group Name Lookups.948gitlab # [ 8.654429] systemd[1]: Starting User Login Management...949gitlab # [ 8.679880] systemd[1]: Finished resolvconf update.950gitlab # [ 8.681406] systemd[1]: Reached target Preparation for Network.951gitlab # [ 8.683306] systemd[1]: Starting DHCP Client...952gitlab # [ 8.686906] systemd[1]: Starting Address configuration of eth1...953gitlab # [ 8.688760] systemd[1]: Starting Extra networking commands....954gitlab # [ 8.719984] network-addresses-eth1-start[1166]: adding address 192.168.1.1/24... done955gitlab # [ 8.724357] network-addresses-eth1-start[1166]: adding address 2001:db8:1::1/64... done956gitlab # [ 8.732923] systemd-logind[1161]: New seat seat0.957gitlab # [ 8.736236] systemd-logind[1161]: Watching system buttons on /dev/input/event2 (Power Button)958gitlab # [ 8.738544] systemd-logind[1161]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)959gitlab # [ 8.743328] systemd-logind[1161]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)960gitlab # [ 8.747113] systemd[1]: Finished Address configuration of eth1.961gitlab # [ 8.748741] systemd[1]: Started User Login Management.962gitlab # [ 8.749960] systemd[1]: Starting linger-users.service...963gitlab # [ 8.766923] systemd[1]: linger-users.service: Deactivated successfully.964gitlab # [ 8.769512] systemd[1]: Finished linger-users.service.965gitlab # [ 8.783561] dhcpcd[1183]: dhcpcd-10.3.2 starting966gitlab # [ 8.784924] systemd[1]: Finished Extra networking commands..967gitlab # [ 8.786163] systemd[1]: Reached target Network.968gitlab # [ 8.787369] systemd[1]: Starting gitlab-workhorse.service...969gitlab # [ 8.790509] dhcpcd[1231]: dev: loaded udev970gitlab # [ 8.792904] systemd[1]: Starting Nginx Web Server...971gitlab # [ 8.797440] systemd[1]: Starting PostgreSQL Server...972gitlab # [ 9.063153] 8021q: 802.1Q VLAN Support v1.8973gitlab # [ 9.063988] 8021q: adding VLAN 0 to HW filter on device eth1974gitlab # [ 8.803509] systemd[1]: Starting Redis Server - redis-gitlab...975gitlab # [ 8.808149] systemd[1]: Starting Permit User Sessions...976gitlab # [ 8.825908] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.977gitlab # [ 8.836628] systemd[1]: Finished Permit User Sessions.978gitlab # [ 8.841345] systemd[1]: Started Getty on tty1.979gitlab # [ 8.844841] systemd[1]: Reached target Login Prompts.980gitlab # [ 9.161957] cfg80211: Loading compiled-in X.509 certificates for regulatory database981gitlab # [ 9.168599] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'982gitlab # [ 9.170231] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'983gitlab # [ 9.173162] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2984gitlab # [ 9.175368] cfg80211: failed to load regulatory.db985gitlab # [ 9.200469] 8021q: adding VLAN 0 to HW filter on device eth0986gitlab # [ 8.938366] dhcpcd[1231]: eth0: waiting for carrier987gitlab # [ 8.942937] dhcpcd[1231]: eth0: carrier acquired988gitlab # [ 8.944496] nginx-pre-start[1256]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok989gitlab # [ 8.948096] nginx-pre-start[1256]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful990gitlab # [ 8.954276] systemd[1]: Started Nginx Web Server.991gitlab # [ 8.955921] dhcpcd[1231]: DUID 00:01:00:01:31:dc:ec:e8:52:54:00:12:34:56992gitlab # [ 8.958420] dhcpcd[1231]: eth0: IAID 00:12:34:56993gitlab # [ 8.960401] dhcpcd[1231]: eth0: adding address fe80::5054:ff:fe12:3456994gitlab # [ 8.973563] postgresql-pre-start[1261]: The files belonging to this database system will be owned by user "postgres".995gitlab # [ 8.975195] postgresql-pre-start[1261]: This user must also own the server process.996gitlab # [ 8.979937] postgresql-pre-start[1261]: The database cluster will be initialized with locale "en_US.UTF-8".997gitlab # [ 8.980887] postgresql-pre-start[1261]: The default database encoding has accordingly been set to "UTF8".998gitlab # [ 8.982160] postgresql-pre-start[1261]: The default text search configuration will be set to "english".999gitlab # [ 8.983416] postgresql-pre-start[1261]: Data page checksums are disabled.1000gitlab # [ 8.984406] postgresql-pre-start[1261]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1001gitlab # [ 8.985308] postgresql-pre-start[1261]: creating subdirectories ... ok1002gitlab # [ 8.986181] postgresql-pre-start[1261]: selecting dynamic shared memory implementation ... posix1003gitlab # [ 8.996416] redis-server[1255]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1004gitlab # [ 9.000333] redis-server[1255]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1005gitlab # [ 9.002639] redis-server[1255]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1255, just started1006gitlab # [ 9.005792] systemd[1]: Started Redis Server - redis-gitlab.1007gitlab # [ 9.007581] redis-server[1255]: Configuration loaded1008gitlab # [ 9.009326] redis[1255]: monotonic clock: POSIX clock_gettime1009gitlab # [ 9.011275] redis[1255]: Running mode=standalone, port=0.1010gitlab # [ 9.013561] redis[1255]: Server initialized1011gitlab # [ 9.014753] redis[1255]: Ready to accept connections unix1012gitlab # [ 9.015923] redis[1255]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1013gitlab # [ 9.039128] postgresql-pre-start[1261]: selecting default "max_connections" ... 1001014gitlab # [ 9.082090] postgresql-pre-start[1261]: selecting default "shared_buffers" ... 128MB1015gitlab # [ 9.190523] dhcpcd[1231]: eth0: soliciting a DHCP lease1016gitlab # [ 9.461665] NET: Registered PF_PACKET protocol family1017gitlab # [ 9.205294] dhcpcd[1231]: eth0: offered 10.0.2.15 from 10.0.2.21018gitlab # [ 9.209301] dhcpcd[1231]: eth0: probing address 10.0.2.15/241019gitlab # [ 9.275967] systemd[1]: Started gitlab-workhorse.service.1020gitlab # [ 9.487929] postgresql-pre-start[1261]: selecting default time zone ... UTC1021gitlab # [ 9.490324] postgresql-pre-start[1261]: creating configuration files ... ok1022gitlab # [ 9.524245] gitlab-workhorse[1292]: time="2026-07-05T10:22:33Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41023gitlab # [ 9.526374] gitlab-workhorse[1292]: time="2026-07-05T10:22:33Z" level=info msg="Using redis/go-redis"1024gitlab # [ 9.527516] gitlab-workhorse[1292]: time="2026-07-05T10:22:33Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1025gitlab # [ 9.663326] postgresql-pre-start[1261]: running bootstrap script ... ok1026gitlab # [ 10.119556] postgresql-pre-start[1261]: performing post-bootstrap initialization ... ok1027gitlab # [ 10.433958] postgresql-pre-start[1261]: syncing data to disk ... ok1028gitlab # [ 10.438436] postgresql-pre-start[1261]: initdb: warning: enabling "trust" authentication for local connections1029gitlab # [ 10.444615] postgresql-pre-start[1261]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1030gitlab # [ 10.453993] postgresql-pre-start[1261]: Success. You can now start the database server using:1031gitlab # [ 10.459213] postgresql-pre-start[1261]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1032gitlab # [ 10.542780] postgres[1316]: [1316] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1033gitlab # [ 10.544266] postgres[1316]: [1316] LOG: listening on IPv6 address "::1", port 54321034gitlab # [ 10.546930] postgres[1316]: [1316] LOG: listening on IPv4 address "127.0.0.1", port 54321035gitlab # [ 10.561234] postgres[1316]: [1316] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1036gitlab # [ 10.576763] postgres[1323]: [1323] LOG: database system was shut down at 2026-07-05 10:22:33 GMT1037gitlab # [ 10.589315] postgres[1316]: [1316] LOG: database system is ready to accept connections1038gitlab # [ 10.596375] systemd[1]: Started PostgreSQL Server.1039gitlab # [ 10.599506] systemd[1]: Starting PostgreSQL Setup Scripts...1040gitlab # [ 10.778779] postgresql-setup-start[1334]: CREATE DATABASE1041gitlab # [ 10.799282] postgresql-setup-start[1348]: CREATE ROLE1042gitlab # [ 10.809449] postgresql-setup-start[1350]: ALTER DATABASE1043gitlab # [ 10.831230] postgresql-setup-start[1355]: CREATE ROLE1044gitlab # [ 10.833975] systemd[1]: Finished PostgreSQL Setup Scripts.1045gitlab # [ 10.835111] systemd[1]: Reached target PostgreSQL.1046gitlab # [ 10.836833] systemd[1]: Starting gitlab-postgresql.service...1047gitlab # [ 10.951245] gitlab-postgresql-start[1363]: CREATE DATABASE1048gitlab # [ 11.010970] gitlab-postgresql-start[1372]: CREATE EXTENSION1049gitlab # [ 11.071719] gitlab-postgresql-start[1374]: CREATE EXTENSION1050gitlab # [ 11.073189] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1368]: mkdir -p /var/gitlab/state/home/.ssh: OK1051gitlab # [ 11.075630] systemd[1]: Finished gitlab-postgresql.service.1052gitlab # [ 11.078601] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1368]: chmod 700 /var/gitlab/state/home/.ssh: OK1053gitlab # [ 11.128511] dhcpcd[1231]: eth0: soliciting an IPv6 router1054gitlab # [ 11.130280] dhcpcd[1231]: eth0: Router Advertisement from fe80::21055gitlab # [ 11.131868] dhcpcd[1231]: eth0: adding address fec0::5054:ff:fe12:3456/641056gitlab # [ 11.133562] dhcpcd[1231]: eth0: adding route to fec0::/641057gitlab # [ 11.135276] dhcpcd[1231]: eth0: adding default route via fe80::21058gitlab # [ 11.146620] systemd[1]: Finished gitlab-config.service.1059gitlab # [ 11.153247] systemd[1]: Started gitaly.service.1060gitlab # [ 11.157707] systemd[1]: Starting gitlab-db-config.service...1061gitlab # [ 11.755445] gitaly[1389]: time="2026-07-05T10:22:35.632Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13891062gitlab # [ 11.759611] gitaly[1389]: time="2026-07-05T10:22:35.632Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13891063gitlab # [ 11.764536] gitaly[1389]: time="2026-07-05T10:22:35.632Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13891064gitlab # [ 11.769114] gitaly[1389]: time="2026-07-05T10:22:35.633Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13891065gitlab # [ 11.772543] gitaly[1389]: time="2026-07-05T10:22:35.633Z" level=info msg="Starting Gitaly" pid=1389 version=18.11.41066gitlab # [ 11.940129] gitaly[1389]: time="2026-07-05T10:22:35.818Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=184 pid=13891067gitlab # [ 11.941231] gitaly[1389]: time="2026-07-05T10:22:35.818Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13891068gitlab # [ 11.941669] gitaly[1389]: time="2026-07-05T10:22:35.818Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13891069gitlab # [ 11.943177] gitaly[1389]: time="2026-07-05T10:22:35.818Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1389/git-exec-2414896742.d/git pid=13891070gitlab # [ 11.953258] gitaly[1389]: time="2026-07-05T10:22:35.831Z" level=info msg="finished detecting git version" duration_ms=12 pid=13891071gitlab # [ 11.954491] gitaly[1389]: time="2026-07-05T10:22:35.831Z" level=info msg="using Git version" pid=1389 version=2.53.g1072gitlab # [ 11.955550] gitaly[1389]: time="2026-07-05T10:22:35.833Z" level=info msg="clearing disk cache object folder" pid=1389 storage=default1073gitlab # [ 11.956953] gitaly[1389]: time="2026-07-05T10:22:35.833Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4058798769 pid=1389 storage=default1074gitlab # [ 11.958072] gitaly[1389]: time="2026-07-05T10:22:35.833Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4058798769 pid=1389 storage=default1075gitlab # [ 11.959245] gitaly[1389]: time="2026-07-05T10:22:35.833Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13891076gitlab # [ 11.960302] gitaly[1389]: time="2026-07-05T10:22:35.834Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13891077gitlab # [ 11.960982] gitaly[1389]: time="2026-07-05T10:22:35.834Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4058798769 clear_duration_ms=0 pid=1389 storage=default1078gitlab # [ 12.061241] gitaly[1389]: time="2026-07-05T10:22:35.938Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13891079gitlab # [ 12.062510] gitaly[1389]: time="2026-07-05T10:22:35.939Z" level=info msg="counting repositories" pid=13891080gitlab # [ 12.063276] gitaly[1389]: time="2026-07-05T10:22:35.939Z" level=info msg="finished tempdir cleaner walk" pid=1389 storage=default time_ms=01081gitlab # [ 12.064394] gitaly[1389]: time="2026-07-05T10:22:35.939Z" level=info msg="starting to count repositories" pid=1389 storage_path=/var/gitlab/state/repositories1082gitlab # [ 12.065871] gitaly[1389]: time="2026-07-05T10:22:35.939Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1389 storage_path=/var/gitlab/state/repositories1083gitlab # [ 12.067204] gitaly[1389]: time="2026-07-05T10:22:35.939Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13891084gitlab # [ 12.067861] gitaly[1389]: time="2026-07-05T10:22:35.939Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1389 schema=unix1085gitlab # [ 12.085723] gitaly[1389]: time="2026-07-05T10:22:35.963Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1389/sock.d/intern pid=1389 schema=unix1086gitlab # [ 12.091815] gitaly[1389]: time="2026-07-05T10:22:35.969Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13891087gitlab # [ 12.092990] gitaly[1389]: time="2026-07-05T10:22:35.969Z" level=info msg="Started Gitaly" duration_ms=336 pid=13891088gitlab # [ 12.095887] gitaly[1389]: time="2026-07-05T10:22:35.970Z" level=info msg="maintenance: daily scheduled" pid=1389 scheduled="2026-07-05 12:00:00 +0000 UTC"1089gitlab # [ 12.524921] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1392]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1090gitlab # [ 12.529811] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1392]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1091gitlab # [ 12.563907] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1392]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1092gitlab # [ 13.989462] gitaly[1389]: time="2026-07-05T10:22:37.867Z" level=info msg="License database preloaded" duration_ms=2233 pid=13891093gitlab # [ 14.846346] dhcpcd[1231]: eth0: leased 10.0.2.15 for 86400 seconds1094gitlab # [ 14.850355] dhcpcd[1231]: eth0: adding route to 10.0.2.0/241095gitlab # [ 14.853236] dhcpcd[1231]: eth0: adding default route via 10.0.2.21096gitlab # [ 14.982499] systemd[1]: Started DHCP Client.1097gitlab # [ 14.986850] systemd[1]: Reached target Network is Online.1098gitlab # [ 42.803753] postgres[1741]: [1741] ERROR: relation "feature_gates" does not exist at character 5231099gitlab # [ 42.806388] postgres[1741]: [1741] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1100gitlab # [ 42.809996] postgres[1741]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1101gitlab # [ 42.813837] postgres[1741]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1102gitlab # [ 42.817822] postgres[1741]: attidentity AS identity,1103gitlab # [ 42.821762] postgres[1741]: attgenerated as attgenerated1104gitlab # [ 42.825229] postgres[1741]: FROM pg_attribute a1105gitlab # [ 42.828487] postgres[1741]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1106gitlab # [ 42.832795] postgres[1741]: LEFT JOIN pg_type t ON a.atttypid = t.oid1107gitlab # [ 42.835456] postgres[1741]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1108gitlab # [ 42.838117] postgres[1741]: WHERE a.attrelid = '"feature_gates"'::regclass1109gitlab # [ 42.840731] postgres[1741]: AND a.attnum > 0 AND NOT a.attisdropped1110gitlab # [ 42.842914] postgres[1741]: ORDER BY a.attnum1111gitlab # [ 42.844871] postgres[1741]:1112gitlab # [ 43.001597] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1392]: W, [2026-07-05T10:23:06.878669 #1392] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1113gitlab # [ 48.474427] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1803]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1114gitlab # [ 48.483322] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1803]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1115gitlab # [ 49.590285] gitlab-workhorse[1292]: time="2026-07-05T10:23:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1116gitlab # [ 59.599725] gitlab-workhorse[1292]: time="2026-07-05T10:23:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1117gitlab # [ 65.228511] postgres[1878]: [1878] ERROR: relation "table_without_models" does not exist1118gitlab # [ 65.230579] postgres[1878]: [1878] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1119gitlab # [ 69.609148] gitlab-workhorse[1292]: time="2026-07-05T10:23:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1120gitlab # [ 78.966805] systemd[1]: Finished gitlab-db-config.service.1121gitlab # [ 78.972270] systemd[1]: Started gitlab-sidekiq.service.1122gitlab # [ 78.976449] systemd[1]: Starting gitlab.service...1123gitlab # [ 79.316988] bundle[2072]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1124gitlab # [ 79.319604] bundle[2072]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1125gitlab # [ 79.325236] bundle[2072]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1126gitlab # [ 79.619582] gitlab-workhorse[1292]: time="2026-07-05T10:23:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1127gitlab # [ 82.224645] sidekiq-cluster[2071]: {"severity":"INFO","time":"2026-07-05T10:23:46.084Z","message":"Starting cluster with 1 processes"}1128gitlab # [ 82.348915] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.206Z","pid":2072,"message":"* Enabling systemd notification integration"}1129gitlab # [ 82.351223] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.225Z","pid":2072,"message":"Puma starting in cluster mode..."}1130gitlab # [ 82.352589] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.226Z","pid":2072,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1131gitlab # [ 82.354492] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.226Z","pid":2072,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1132gitlab # [ 82.356253] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.226Z","pid":2072,"message":"* Min threads: 0"}1133gitlab # [ 82.357609] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.226Z","pid":2072,"message":"* Max threads: 4"}1134gitlab # [ 82.359415] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.226Z","pid":2072,"message":"* Environment: production"}1135gitlab # [ 82.361176] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.226Z","pid":2072,"message":"* Master PID: 2072"}1136gitlab # [ 82.362538] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.226Z","pid":2072,"message":"* Workers: 2"}1137gitlab # [ 82.364168] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.226Z","pid":2072,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1138gitlab # [ 82.365580] bundle[2072]: {"timestamp":"2026-07-05T10:23:46.226Z","pid":2072,"message":"* Preloading application"}1139gitlab # [ 89.629556] gitlab-workhorse[1292]: time="2026-07-05T10:23:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1140gitlab # [ 99.631773] gitlab-workhorse[1292]: time="2026-07-05T10:24:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1141gitlab # [ 106.037313] bundle[2072]: W, [2026-07-05T10:24:09.914011 #2072] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1142gitlab # [ 108.833821] sidekiq-cluster[2102]: 2026-07-05T10:24:12.710Z pid=2102 tid=tz2 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2102, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431669248}1143gitlab # [ 109.025183] sidekiq-cluster[2102]: W, [2026-07-05T10:24:12.902881 #2102] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1144gitlab # [ 109.108253] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:12.985Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1145gitlab # [ 109.641499] gitlab-workhorse[1292]: time="2026-07-05T10:24:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 111.087824] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:14.964Z","message":"GitLab reliable fetch activated!"}1147gitlab # [ 111.089883] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:14.964Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1148gitlab # [ 111.144185] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.021Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1149gitlab # [ 111.154573] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.032Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1150gitlab # [ 111.164386] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.042Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1151gitlab # [ 111.175272] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.052Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1152gitlab # [ 111.185552] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.063Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1153gitlab # [ 111.194659] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.072Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1154gitlab # [ 111.203399] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.081Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1155gitlab # [ 111.212150] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.089Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1156gitlab # [ 111.220636] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.098Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1157gitlab # [ 111.229663] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.107Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1158gitlab # [ 111.238682] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.116Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1159gitlab # [ 111.250237] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.128Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1160gitlab # [ 111.257777] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.135Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1161gitlab # [ 111.265754] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.143Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1162gitlab # [ 111.273381] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.151Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1163gitlab # [ 111.282354] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.160Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1164gitlab # [ 111.289711] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.167Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1165gitlab # [ 111.297751] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.175Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1166gitlab # [ 111.305487] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.183Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1167gitlab # [ 111.312932] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.190Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1168gitlab # [ 111.320506] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.198Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1169gitlab # [ 111.328319] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.206Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1170gitlab # [ 111.335222] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.213Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1171gitlab # [ 111.341943] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.219Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1172gitlab # [ 111.348672] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.226Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1173gitlab # [ 111.353140] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.231Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1174gitlab # [ 111.358199] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.236Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1175gitlab # [ 111.363332] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.241Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1176gitlab # [ 111.367299] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.244Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1177gitlab # [ 111.371487] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.247Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1178gitlab # [ 111.376246] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.254Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1179gitlab # [ 111.379837] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.257Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1180gitlab # [ 111.383542] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.261Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1181gitlab # [ 111.386829] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.264Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1182gitlab # [ 111.390665] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.268Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1183gitlab # [ 111.394195] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.272Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1184gitlab # [ 111.397549] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.275Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1185gitlab # [ 111.401209] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.279Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1186gitlab # [ 111.404743] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.282Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1187gitlab # [ 111.407931] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.286Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1188gitlab # [ 111.411522] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.289Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1189gitlab # [ 111.415194] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.293Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1190gitlab # [ 111.418509] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.296Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1191gitlab # [ 111.421422] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.299Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1192gitlab # [ 111.424580] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.302Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1193gitlab # [ 111.428392] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.306Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1194gitlab # [ 111.431688] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.309Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1195gitlab # [ 111.435244] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.313Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1196gitlab # [ 111.438550] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.316Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1197gitlab # [ 111.441578] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.319Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1198gitlab # [ 111.444993] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.323Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1199gitlab # [ 111.448426] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.326Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1200gitlab # [ 111.452427] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.330Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1201gitlab # [ 111.462608] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.340Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1202gitlab # [ 111.466678] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.344Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1203gitlab # [ 111.470633] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.348Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1204gitlab # [ 111.474406] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.352Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1205gitlab # [ 111.477768] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.355Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1206gitlab # [ 111.481523] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.359Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1207gitlab # [ 111.484631] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.362Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1208gitlab # [ 111.488043] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.366Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1209gitlab # [ 111.491102] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.369Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1210gitlab # [ 111.494222] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.372Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1211gitlab # [ 111.497691] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.375Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1212gitlab # [ 111.501371] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.379Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1213gitlab # [ 111.504658] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.382Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1214gitlab # [ 111.508632] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.386Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1215gitlab # [ 111.512069] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.390Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1216gitlab # [ 111.514945] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.393Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1217gitlab # [ 111.518186] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.396Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1218gitlab # [ 111.521039] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.399Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1219gitlab # [ 111.523983] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.402Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1220gitlab # [ 111.526883] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.404Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1221gitlab # [ 111.530155] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.408Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1222gitlab # [ 111.533300] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.411Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1223gitlab # [ 111.536188] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.414Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1224gitlab # [ 111.539201] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.417Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1225gitlab # [ 111.542499] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.420Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1226gitlab # [ 111.545574] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.423Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1227gitlab # [ 111.548964] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.427Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1228gitlab # [ 111.552600] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.430Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1229gitlab # [ 111.555858] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.433Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1230gitlab # [ 111.559158] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.437Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1231gitlab # [ 111.562949] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.441Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1232gitlab # [ 111.566442] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.444Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1233gitlab # [ 111.569829] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.447Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1234gitlab # [ 111.572861] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.450Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1235gitlab # [ 111.576531] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.454Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1236gitlab # [ 111.579301] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.457Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1237gitlab # [ 111.582508] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.460Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1238gitlab # [ 111.585796] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.463Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1239gitlab # [ 111.606431] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.484Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1240gitlab # [ 111.610045] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.488Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1241gitlab # [ 111.613547] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.491Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1242gitlab # [ 111.616964] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.495Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1243gitlab # [ 111.621562] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.499Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1244gitlab # [ 111.625531] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.503Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1245gitlab # [ 111.629709] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.507Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1246gitlab # [ 111.632898] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.510Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1247gitlab # [ 111.644241] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.522Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1248gitlab # [ 111.647191] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.525Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1249gitlab # [ 111.650583] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.528Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1250gitlab # [ 111.654893] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.532Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1251gitlab # [ 111.658543] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.536Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1252gitlab # [ 111.662545] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.540Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1253gitlab # [ 111.666948] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.545Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1254gitlab # [ 111.670168] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.548Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1255gitlab # [ 111.673068] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.551Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1256gitlab # [ 111.677102] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.555Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1257gitlab # [ 111.680486] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:15.558Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1258gitlab # [ 119.642948] gitlab-workhorse[1292]: time="2026-07-05T10:24:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1259gitlab # [ 129.645777] gitlab-workhorse[1292]: time="2026-07-05T10:24:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1260gitlab # [ 132.501413] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:36.378Z","message":"Booted Rails 7.2.3 application in production environment"}1261gitlab # [ 132.503707] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:36.379Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1262gitlab # [ 132.505608] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:36.379Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1263gitlab # [ 132.506924] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:36.379Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1264gitlab # [ 132.516518] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:36.394Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1265gitlab # [ 133.815165] bundle[2072]: {"timestamp":"2026-07-05T10:24:37.692Z","pid":2072,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1266gitlab # [ 133.822856] bundle[2072]: {"timestamp":"2026-07-05T10:24:37.692Z","pid":2072,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1267gitlab # [ 133.829960] bundle[2072]: {"timestamp":"2026-07-05T10:24:37.692Z","pid":2072,"message":"! #\u003cThread:0x00007568aa418790 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1268gitlab # [ 133.847124] bundle[2072]: {"timestamp":"2026-07-05T10:24:37.692Z","pid":2072,"message":"! #\u003cThread:0x00007568aa631478 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1269gitlab # [ 133.859269] bundle[2072]: {"timestamp":"2026-07-05T10:24:37.692Z","pid":2072,"message":"Use Ctrl-C to stop"}1270gitlab # [ 133.861448] bundle[2072]: {"timestamp":"2026-07-05T10:24:37.693Z","pid":2072,"message":"Sending status to systemd every 1.0 sec"}1271gitlab # [ 133.873723] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:37.751Z","message":"Cleaning working queues"}1272gitlab # [ 133.876352] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:37.753Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4bba43372b859a9845aa4ea9","created_at":"2026-07-05T10:23:42.266Z","trace_propagation_headers":{"sentry-trace":"e8f7c28e62fc4603b7f4dc0f95791e8a-5ff89b98c4b84873","baggage":"sentry-trace_id=e8f7c28e62fc4603b7f4dc0f95791e8a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"02273ec5cbb7135fc43075354704ce59","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-05T10:23:42.281Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4bba43372b859a9845aa4ea9: start","job_status":"start","queue_duration_s":55.471301,"scheduling_latency_s":55.471353}1273gitlab # [ 134.034573] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:24:37.912Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4bba43372b859a9845aa4ea9","created_at":"2026-07-05T10:23:42.266Z","trace_propagation_headers":{"sentry-trace":"e8f7c28e62fc4603b7f4dc0f95791e8a-5ff89b98c4b84873","baggage":"sentry-trace_id=e8f7c28e62fc4603b7f4dc0f95791e8a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"02273ec5cbb7135fc43075354704ce59","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-05T10:23:42.281Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4bba43372b859a9845aa4ea9: done: 0.159356 sec","job_status":"done","queue_duration_s":55.471301,"scheduling_latency_s":55.471353,"redis_calls":26,"redis_duration_s":0.008135,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.004572,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002228,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001335,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08426489899999999,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.159356,"completed_at":"2026-07-05T10:24:37.912Z","load_balancing_strategy":"primary","db_duration_s":0.053247,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1274gitlab # [ 134.170771] bundle[2072]: {"timestamp":"2026-07-05T10:24:38.047Z","pid":2072,"message":"- Worker 0 (PID: 2596) booted in 0.32s, phase: 0"}1275gitlab # [ 134.175186] bundle[2072]: {"timestamp":"2026-07-05T10:24:38.053Z","pid":2072,"message":"- Worker 1 (PID: 2600) booted in 0.31s, phase: 0"}1276gitlab # [ 134.178739] systemd[1]: Started gitlab.service.1277gitlab # [ 134.180183] systemd[1]: Reached target Common target for all GitLab services..1278gitlab # [ 134.182115] systemd[1]: Starting setup-gitlab.service...1279gitlab # [ 134.197929] setup-gitlab-start[2640]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1280gitlab # [ 136.596351] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:24:40 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 23771281gitlab # [ 136.608198] setup-gitlab-start[2659]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1282gitlab # [ 136.612277] setup-gitlab-start[2660]: ++ jq -r .access_token1283gitlab # [ 137.922134] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:24:41 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 12911284gitlab # [ 137.931617] setup-gitlab-start[2640]: + BEARER=11647bc5113bc66e4ef49e8748079ed2ee411c36e9b1e0152ef635dc9b1218e91285gitlab # [ 137.934954] setup-gitlab-start[2640]: + AUTH='Authorization: Bearer 11647bc5113bc66e4ef49e8748079ed2ee411c36e9b1e0152ef635dc9b1218e9'1286gitlab # [ 137.938333] setup-gitlab-start[2640]: + curl -fs -X PUT -H 'Authorization: Bearer 11647bc5113bc66e4ef49e8748079ed2ee411c36e9b1e0152ef635dc9b1218e9' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1287gitlab # [ 138.707129] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:24:42 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 7491288gitlab # [ 138.709833] setup-gitlab-start[2640]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1289gitlab # [ 138.728577] runuser[2682]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1290gitlab # [ 139.101957] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1291gitlab # [ 139.105671] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1292gitlab # [ 139.108443] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1293gitlab # [ 139.676631] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1294gitlab # [ 139.685492] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1295gitlab # [ 139.694186] setup-gitlab-start[2683]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1296gitlab # [ 158.592359] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:02.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6e7b3baac1db6570464205af","created_at":"2026-07-05T10:25:02.446Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.467Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6e7b3baac1db6570464205af: start","job_status":"start","queue_duration_s":0.00256,"scheduling_latency_s":0.00259}1297gitlab # [ 158.737274] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:02.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6e7b3baac1db6570464205af","created_at":"2026-07-05T10:25:02.446Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.467Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6e7b3baac1db6570464205af: done: 0.144961 sec","job_status":"done","queue_duration_s":0.00256,"scheduling_latency_s":0.00259,"redis_calls":19,"redis_duration_s":0.021988999999999998,"redis_read_bytes":21,"redis_write_bytes":4781,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00618,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000691,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.010098,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00502,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000878324,"duration_s":0.144961,"completed_at":"2026-07-05T10:25:02.614Z","load_balancing_strategy":"primary","db_duration_s":0.007429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 158.831294] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:02.620Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"11b7a55168f4f07c6d12f193","created_at":"2026-07-05T10:25:02.529Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"919d140df1bc43b7c9d4d762a3c171c9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.556Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-11b7a55168f4f07c6d12f193: start","job_status":"start","queue_duration_s":0.064622,"scheduling_latency_s":0.064645}1299gitlab # [ 158.853988] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:02.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"11b7a55168f4f07c6d12f193","created_at":"2026-07-05T10:25:02.529Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"919d140df1bc43b7c9d4d762a3c171c9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.556Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-11b7a55168f4f07c6d12f193: done: 0.054544 sec","job_status":"done","queue_duration_s":0.064622,"scheduling_latency_s":0.064645,"redis_calls":10,"redis_duration_s":0.003101,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00055,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0018,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000751,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000398374,"duration_s":0.054544,"completed_at":"2026-07-05T10:25:02.675Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 158.920603] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:02.676Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d89cd9a6e77b18d66cf00c7e","created_at":"2026-07-05T10:25:02.574Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-f8f3ac91c6db43c6","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.603Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d89cd9a6e77b18d66cf00c7e: start","job_status":"start","queue_duration_s":0.072659,"scheduling_latency_s":0.072679}1301gitlab # [ 161.078147] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:04.955Z","message":"No existing merge request to be cleaned up."}1302gitlab # [ 161.118109] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:04.995Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d89cd9a6e77b18d66cf00c7e","created_at":"2026-07-05T10:25:02.574Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-f8f3ac91c6db43c6","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.603Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d89cd9a6e77b18d66cf00c7e: done: 2.319041 sec","job_status":"done","queue_duration_s":0.072659,"scheduling_latency_s":0.072679,"redis_calls":13,"redis_duration_s":0.013418,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008449,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004521,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.489528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001119696,"duration_s":2.319041,"completed_at":"2026-07-05T10:25:04.995Z","load_balancing_strategy":"primary","db_duration_s":0.044994,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 161.122121] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:04.997Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dba561f05e2446a0d5b16337","created_at":"2026-07-05T10:25:02.574Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-f8f3ac91c6db43c6","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.603Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dba561f05e2446a0d5b16337: start","job_status":"start","queue_duration_s":2.393436,"scheduling_latency_s":2.393464}1304gitlab # [ 161.203556] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.047Z","message":"No existing merge request to be cleaned up."}1305gitlab # [ 161.248237] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.125Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dba561f05e2446a0d5b16337","created_at":"2026-07-05T10:25:02.574Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-f8f3ac91c6db43c6","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.603Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dba561f05e2446a0d5b16337: done: 0.128098 sec","job_status":"done","queue_duration_s":2.393436,"scheduling_latency_s":2.393464,"redis_calls":10,"redis_duration_s":0.047728999999999994,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00237,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.045359,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.036,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.036,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.128098,"completed_at":"2026-07-05T10:25:05.125Z","load_balancing_strategy":"primary","db_duration_s":0.055956,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 161.320256] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.197Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"89b82d479314caa8a57ef924","created_at":"2026-07-05T10:25:02.574Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-f8f3ac91c6db43c6","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.603Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-89b82d479314caa8a57ef924: start","job_status":"start","queue_duration_s":2.593818,"scheduling_latency_s":2.593856}1307gitlab # [ 161.385192] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.262Z","message":"No existing merge request to be cleaned up."}1308gitlab # [ 161.422285] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.299Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"89b82d479314caa8a57ef924","created_at":"2026-07-05T10:25:02.574Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-f8f3ac91c6db43c6","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.603Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-89b82d479314caa8a57ef924: done: 0.101444 sec","job_status":"done","queue_duration_s":2.593818,"scheduling_latency_s":2.593856,"redis_calls":10,"redis_duration_s":0.017242,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008118,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.009124,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.101444,"completed_at":"2026-07-05T10:25:05.299Z","load_balancing_strategy":"primary","db_duration_s":0.014204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 161.491171] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.368Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9b803f14e2ba584b86e258b0","created_at":"2026-07-05T10:25:02.574Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-f8f3ac91c6db43c6","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.603Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9b803f14e2ba584b86e258b0: start","job_status":"start","queue_duration_s":2.764481,"scheduling_latency_s":2.764547}1310gitlab # [ 161.558248] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.435Z","message":"No existing merge request to be cleaned up."}1311gitlab # [ 161.595190] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.472Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9b803f14e2ba584b86e258b0","created_at":"2026-07-05T10:25:02.574Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-f8f3ac91c6db43c6","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f4bb732a303b59ec24348a4b6354a6f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.603Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9b803f14e2ba584b86e258b0: done: 0.104231 sec","job_status":"done","queue_duration_s":2.764481,"scheduling_latency_s":2.764547,"redis_calls":10,"redis_duration_s":0.022295,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.009477,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.012818,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.104231,"completed_at":"2026-07-05T10:25:05.472Z","load_balancing_strategy":"primary","db_duration_s":0.016832,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 161.679211] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.556Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"436f620f6d31696ec2240417","created_at":"2026-07-05T10:25:02.573Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a02913c56efb5c79e59da100b7453aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.617Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-436f620f6d31696ec2240417: start","job_status":"start","queue_duration_s":2.938288,"scheduling_latency_s":2.938355}1313gitlab # [ 161.730843] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.608Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1314gitlab # [ 161.754877] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"436f620f6d31696ec2240417","created_at":"2026-07-05T10:25:02.573Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a02913c56efb5c79e59da100b7453aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.617Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-436f620f6d31696ec2240417: done: 0.076515 sec","job_status":"done","queue_duration_s":2.938288,"scheduling_latency_s":2.938355,"redis_calls":10,"redis_duration_s":0.022855,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004707,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014284,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003864,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002020927,"duration_s":0.076515,"completed_at":"2026-07-05T10:25:05.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002277,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 161.769792] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.644Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"04e398c229f2fdf9df565e9e","created_at":"2026-07-05T10:25:02.638Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aac19dbb607acdc2340e77f9d849b2d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-05T10:25:02.681Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-04e398c229f2fdf9df565e9e: start","job_status":"start","queue_duration_s":2.963197,"scheduling_latency_s":2.963241}1316gitlab # [ 161.813698] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.691Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1317gitlab # [ 161.834571] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.712Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"04e398c229f2fdf9df565e9e","created_at":"2026-07-05T10:25:02.638Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aac19dbb607acdc2340e77f9d849b2d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-05T10:25:02.681Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-04e398c229f2fdf9df565e9e: done: 0.067561 sec","job_status":"done","queue_duration_s":2.963197,"scheduling_latency_s":2.963241,"redis_calls":10,"redis_duration_s":0.022997999999999998,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00214,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004408,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003000102,"duration_s":0.067561,"completed_at":"2026-07-05T10:25:05.712Z","load_balancing_strategy":"primary","db_duration_s":0.005847,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 161.846103] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.714Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e65936c49bebe1f25f3ce8b1","created_at":"2026-07-05T10:25:02.699Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"093c015592059c5b7db8bd39671fb058","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-05T10:25:02.708Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e65936c49bebe1f25f3ce8b1: start","job_status":"start","queue_duration_s":3.005497,"scheduling_latency_s":3.005518}1319gitlab # [ 161.914812] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.792Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1320gitlab # [ 161.953863] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.831Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e65936c49bebe1f25f3ce8b1","created_at":"2026-07-05T10:25:02.699Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"093c015592059c5b7db8bd39671fb058","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-05T10:25:02.708Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e65936c49bebe1f25f3ce8b1: done: 0.117277 sec","job_status":"done","queue_duration_s":3.005497,"scheduling_latency_s":3.005518,"redis_calls":10,"redis_duration_s":0.037009,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.008186,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.026142,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001287873,"duration_s":0.117277,"completed_at":"2026-07-05T10:25:05.831Z","load_balancing_strategy":"primary","db_duration_s":0.059241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 162.030938] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:05.908Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"7a3fd87951c11edbdb1191bd","created_at":"2026-07-05T10:25:02.723Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6878c33b0634f9eea54ad4ac32273ebb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.735Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-7a3fd87951c11edbdb1191bd: start","job_status":"start","queue_duration_s":3.172329,"scheduling_latency_s":3.172369}1322gitlab # [ 162.287315] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"7a3fd87951c11edbdb1191bd","created_at":"2026-07-05T10:25:02.723Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6878c33b0634f9eea54ad4ac32273ebb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.735Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-7a3fd87951c11edbdb1191bd: done: 0.256055 sec","job_status":"done","queue_duration_s":3.172329,"scheduling_latency_s":3.172369,"redis_calls":14,"redis_duration_s":0.032898,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.009397,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012737,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.010764,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002240787,"exclusive_lock_hold_duration_s":0.150958851,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.256055,"completed_at":"2026-07-05T10:25:06.163Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.114938,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 162.293301] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.171Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"7642a0f2541431dcec1321fb","created_at":"2026-07-05T10:25:02.754Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d216a6ff7dfcbcb8fc74f5daa9fd36c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-05T10:25:02.763Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-7642a0f2541431dcec1321fb: start","job_status":"start","queue_duration_s":3.407628,"scheduling_latency_s":3.407665}1324gitlab # [ 162.368329] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.199Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"7642a0f2541431dcec1321fb","created_at":"2026-07-05T10:25:02.754Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d216a6ff7dfcbcb8fc74f5daa9fd36c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-05T10:25:02.763Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-7642a0f2541431dcec1321fb: done: 0.028416 sec","job_status":"done","queue_duration_s":3.407628,"scheduling_latency_s":3.407665,"redis_calls":10,"redis_duration_s":0.002487,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000481,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001538,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000468,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000397257,"duration_s":0.028416,"completed_at":"2026-07-05T10:25:06.199Z","load_balancing_strategy":"primary","db_duration_s":0.002706,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 162.380198] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.200Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6088411a5ee22f8a7c44cf7c","created_at":"2026-07-05T10:25:02.773Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55f835c1f95da881c890b87fd3d26f20","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.846Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6088411a5ee22f8a7c44cf7c: start","job_status":"start","queue_duration_s":3.354501,"scheduling_latency_s":3.354522}1326gitlab # [ 162.458468] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.222Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6088411a5ee22f8a7c44cf7c","created_at":"2026-07-05T10:25:02.773Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55f835c1f95da881c890b87fd3d26f20","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.846Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6088411a5ee22f8a7c44cf7c: done: 0.021477 sec","job_status":"done","queue_duration_s":3.354501,"scheduling_latency_s":3.354522,"redis_calls":12,"redis_duration_s":0.003977,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000727,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001391,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000574654,"exclusive_lock_hold_duration_s":0.003629791,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.021477,"completed_at":"2026-07-05T10:25:06.222Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001831,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 162.470814] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.223Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5cb2a377e8a89f4fe774dd24","created_at":"2026-07-05T10:25:02.862Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"795208f37e9755c5848ed54e46965324","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.885Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5cb2a377e8a89f4fe774dd24: start","job_status":"start","queue_duration_s":3.338472,"scheduling_latency_s":3.338492}1328gitlab # [ 162.555943] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5cb2a377e8a89f4fe774dd24","created_at":"2026-07-05T10:25:02.862Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"795208f37e9755c5848ed54e46965324","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.885Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5cb2a377e8a89f4fe774dd24: done: 0.117949 sec","job_status":"done","queue_duration_s":3.338472,"scheduling_latency_s":3.338492,"redis_calls":12,"redis_duration_s":0.005247,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000653,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00245,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002144,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.088,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001099023,"exclusive_lock_hold_duration_s":0.009539201,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.117949,"completed_at":"2026-07-05T10:25:06.341Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.08959,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 162.563138] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.344Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"0ab78d3541166d2964776f43","created_at":"2026-07-05T10:25:02.897Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"100c5604ba11168410035367053ac215","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.917Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-0ab78d3541166d2964776f43: start","job_status":"start","queue_duration_s":3.426399,"scheduling_latency_s":3.426422}1330gitlab # [ 162.644633] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"0ab78d3541166d2964776f43","created_at":"2026-07-05T10:25:02.897Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"100c5604ba11168410035367053ac215","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.917Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-0ab78d3541166d2964776f43: done: 0.046235 sec","job_status":"done","queue_duration_s":3.426399,"scheduling_latency_s":3.426422,"redis_calls":17,"redis_duration_s":0.007693,"redis_read_bytes":15,"redis_write_bytes":2007,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.004058,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1150,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002234,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000551,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00085,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000756242,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.046235,"completed_at":"2026-07-05T10:25:06.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 162.654961] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.393Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d3d51a95cdd9446c6f30695f","created_at":"2026-07-05T10:25:02.934Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f4be581b35936238d52c7580c38e9de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.950Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d3d51a95cdd9446c6f30695f: start","job_status":"start","queue_duration_s":3.442871,"scheduling_latency_s":3.442912}1332gitlab # [ 162.741148] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.410Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1333gitlab # [ 162.742466] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d3d51a95cdd9446c6f30695f","created_at":"2026-07-05T10:25:02.934Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f4be581b35936238d52c7580c38e9de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.950Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d3d51a95cdd9446c6f30695f: done: 0.023985 sec","job_status":"done","queue_duration_s":3.442871,"scheduling_latency_s":3.442912,"redis_calls":10,"redis_duration_s":0.002628,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000469,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001511,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000648,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000370438,"duration_s":0.023985,"completed_at":"2026-07-05T10:25:06.416Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000557,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 162.750972] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"7f5ab5bca4abb47d7a4fffda","created_at":"2026-07-05T10:25:02.967Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"286deff9639e7a12dd4fc047ac77ed09","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.994Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-7f5ab5bca4abb47d7a4fffda: start","job_status":"start","queue_duration_s":3.42325,"scheduling_latency_s":3.42332}1335gitlab # [ 162.772159] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"7f5ab5bca4abb47d7a4fffda","created_at":"2026-07-05T10:25:02.967Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"286deff9639e7a12dd4fc047ac77ed09","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:02.994Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-7f5ab5bca4abb47d7a4fffda: done: 0.024344 sec","job_status":"done","queue_duration_s":3.42325,"scheduling_latency_s":3.42332,"redis_calls":10,"redis_duration_s":0.002353,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000553,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000329371,"duration_s":0.024344,"completed_at":"2026-07-05T10:25:06.442Z","load_balancing_strategy":"primary","db_duration_s":0.003825,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 162.862130] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.443Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f2dda4fbd61bba481317f42e","created_at":"2026-07-05T10:25:03.017Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1da947c349aad28749292349b8ba728e","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.035Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f2dda4fbd61bba481317f42e: start","job_status":"start","queue_duration_s":3.408318,"scheduling_latency_s":3.408339}1337gitlab # [ 163.007471] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f2dda4fbd61bba481317f42e","created_at":"2026-07-05T10:25:03.017Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1da947c349aad28749292349b8ba728e","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.035Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f2dda4fbd61bba481317f42e: done: 0.441101 sec","job_status":"done","queue_duration_s":3.408318,"scheduling_latency_s":3.408339,"redis_calls":574,"redis_duration_s":0.13051200000000002,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.128035,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.351231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000509841,"duration_s":0.441101,"completed_at":"2026-07-05T10:25:06.884Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00035,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 163.098046] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:06.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7af404a169fe3a54c0575498","created_at":"2026-07-05T10:25:03.048Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3dfbe8cd02736d1169ddbd69747ad7b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.077Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7af404a169fe3a54c0575498: start","job_status":"start","queue_duration_s":3.897662,"scheduling_latency_s":3.897729}1339gitlab # [ 163.152745] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.030Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7af404a169fe3a54c0575498","created_at":"2026-07-05T10:25:03.048Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3dfbe8cd02736d1169ddbd69747ad7b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.077Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7af404a169fe3a54c0575498: done: 0.054809 sec","job_status":"done","queue_duration_s":3.897662,"scheduling_latency_s":3.897729,"redis_calls":12,"redis_duration_s":0.0075910000000000005,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001322,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003527,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002742,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00122753,"exclusive_lock_hold_duration_s":0.009299227,"duration_s":0.054809,"completed_at":"2026-07-05T10:25:07.030Z","load_balancing_strategy":"primary","db_duration_s":0.004128,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 163.155732] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"00a809b5d1260de67d8a4349","created_at":"2026-07-05T10:25:03.102Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76d01b1326b80047b1c63f6376041779","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.133Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-00a809b5d1260de67d8a4349: start","job_status":"start","queue_duration_s":3.899282,"scheduling_latency_s":3.899322}1341gitlab # [ 163.293294] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.169Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"00a809b5d1260de67d8a4349","created_at":"2026-07-05T10:25:03.102Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76d01b1326b80047b1c63f6376041779","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.133Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-00a809b5d1260de67d8a4349: done: 0.136408 sec","job_status":"done","queue_duration_s":3.899282,"scheduling_latency_s":3.899322,"redis_calls":10,"redis_duration_s":0.005231,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001091,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003347,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000793,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.097,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.097,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000969397,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.136408,"completed_at":"2026-07-05T10:25:07.169Z","load_balancing_strategy":"primary","db_duration_s":0.101348,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 163.299593] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.172Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"50e6de5312410f4a38502f1f","created_at":"2026-07-05T10:25:03.164Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f42342a82fbacf090d8d80dc864635f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.182Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-50e6de5312410f4a38502f1f: start","job_status":"start","queue_duration_s":3.989188,"scheduling_latency_s":3.989319}1343gitlab # [ 163.316554] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.194Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"50e6de5312410f4a38502f1f","created_at":"2026-07-05T10:25:03.164Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f42342a82fbacf090d8d80dc864635f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.182Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-50e6de5312410f4a38502f1f: done: 0.022235 sec","job_status":"done","queue_duration_s":3.989188,"scheduling_latency_s":3.989319,"redis_calls":12,"redis_duration_s":0.0027270000000000003,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00087,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001363,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000494,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346133,"duration_s":0.022235,"completed_at":"2026-07-05T10:25:07.194Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000722,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 163.439068] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.195Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3b47c54acdcb7828cb5051b1","created_at":"2026-07-05T10:25:03.198Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a29a11e9d089ebb1b4fb949df0b7e4d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.212Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3b47c54acdcb7828cb5051b1: start","job_status":"start","queue_duration_s":3.983749,"scheduling_latency_s":3.983769}1345gitlab # [ 163.441498] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.207Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1346gitlab # [ 163.444678] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3b47c54acdcb7828cb5051b1","created_at":"2026-07-05T10:25:03.198Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a29a11e9d089ebb1b4fb949df0b7e4d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.212Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3b47c54acdcb7828cb5051b1: done: 0.017778 sec","job_status":"done","queue_duration_s":3.983749,"scheduling_latency_s":3.983769,"redis_calls":10,"redis_duration_s":0.00278,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001803,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000339987,"duration_s":0.017778,"completed_at":"2026-07-05T10:25:07.213Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 163.533570] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6e2233c5388e53b3587ae930","created_at":"2026-07-05T10:25:03.233Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a40a3a7d84fba0dc9e6710152c482f0f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.252Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6e2233c5388e53b3587ae930: start","job_status":"start","queue_duration_s":3.9622,"scheduling_latency_s":3.96222}1348gitlab # [ 163.535074] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6e2233c5388e53b3587ae930","created_at":"2026-07-05T10:25:03.233Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a40a3a7d84fba0dc9e6710152c482f0f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.252Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6e2233c5388e53b3587ae930: done: 0.172838 sec","job_status":"done","queue_duration_s":3.9622,"scheduling_latency_s":3.96222,"redis_calls":12,"redis_duration_s":0.003045,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001416,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000443353,"exclusive_lock_hold_duration_s":0.155079487,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.172838,"completed_at":"2026-07-05T10:25:07.387Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.054136,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 163.612354] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6ec53f0686e72230cef10edd","created_at":"2026-07-05T10:25:03.275Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.306Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6ec53f0686e72230cef10edd: start","job_status":"start","queue_duration_s":4.08243,"scheduling_latency_s":4.082493}1350gitlab # [ 163.613705] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6ec53f0686e72230cef10edd","created_at":"2026-07-05T10:25:03.275Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.306Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6ec53f0686e72230cef10edd: done: 0.092191 sec","job_status":"done","queue_duration_s":4.08243,"scheduling_latency_s":4.082493,"redis_calls":71,"redis_duration_s":0.017017,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002128,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000962,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013243,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000684,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000362616,"duration_s":0.092191,"completed_at":"2026-07-05T10:25:07.481Z","load_balancing_strategy":"primary","db_duration_s":0.001728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 163.692184] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.482Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2f778243a96041c6525ac266","created_at":"2026-07-05T10:25:03.341Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f9a64733773d853d2670101bc67dbcf","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.370Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2f778243a96041c6525ac266: start","job_status":"start","queue_duration_s":4.112683,"scheduling_latency_s":4.112705}1352gitlab # [ 163.693445] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2f778243a96041c6525ac266","created_at":"2026-07-05T10:25:03.341Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f9a64733773d853d2670101bc67dbcf","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.370Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2f778243a96041c6525ac266: done: 0.030334 sec","job_status":"done","queue_duration_s":4.112683,"scheduling_latency_s":4.112705,"redis_calls":13,"redis_duration_s":0.003226,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001654,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01006357,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.030334,"completed_at":"2026-07-05T10:25:07.513Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006818,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1353gitlab # [ 163.769334] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.514Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"716141174378bd75e3621b95","created_at":"2026-07-05T10:25:03.393Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b5528b1fe5aa99eb3ab7becd903d5dcf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.421Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-716141174378bd75e3621b95: start","job_status":"start","queue_duration_s":4.092828,"scheduling_latency_s":4.092848}1354gitlab # [ 163.776743] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.535Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"716141174378bd75e3621b95","created_at":"2026-07-05T10:25:03.393Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b5528b1fe5aa99eb3ab7becd903d5dcf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.421Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-716141174378bd75e3621b95: done: 0.020995 sec","job_status":"done","queue_duration_s":4.092828,"scheduling_latency_s":4.092848,"redis_calls":10,"redis_duration_s":0.002301,"redis_read_bytes":9,"redis_write_bytes":1109,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001413,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000387759,"duration_s":0.020995,"completed_at":"2026-07-05T10:25:07.535Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 163.852414] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4c2d7c2cc3ac79208897c44d","created_at":"2026-07-05T10:25:03.446Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d18661e142bab8db9c64d5f4714b07b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.484Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4c2d7c2cc3ac79208897c44d: start","job_status":"start","queue_duration_s":4.05209,"scheduling_latency_s":4.05211}1356gitlab # [ 163.922178] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.547Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1357gitlab # [ 163.923270] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.552Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4c2d7c2cc3ac79208897c44d","created_at":"2026-07-05T10:25:03.446Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d18661e142bab8db9c64d5f4714b07b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.484Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4c2d7c2cc3ac79208897c44d: done: 0.015805 sec","job_status":"done","queue_duration_s":4.05209,"scheduling_latency_s":4.05211,"redis_calls":10,"redis_duration_s":0.002492,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000537,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000329371,"duration_s":0.015805,"completed_at":"2026-07-05T10:25:07.552Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000351,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 163.941586] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.554Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"42144247032e38f1afc14167","created_at":"2026-07-05T10:25:03.510Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf5f488ecf3382d00a496da2da7be52","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.547Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-42144247032e38f1afc14167: start","job_status":"start","queue_duration_s":4.007269,"scheduling_latency_s":4.007289}1359gitlab # [ 164.022781] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"42144247032e38f1afc14167","created_at":"2026-07-05T10:25:03.510Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-ad383f92cf41402f","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf5f488ecf3382d00a496da2da7be52","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:03.547Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-42144247032e38f1afc14167: done: 0.016259 sec","job_status":"done","queue_duration_s":4.007269,"scheduling_latency_s":4.007289,"redis_calls":10,"redis_duration_s":0.00218,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000511,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000366248,"duration_s":0.016259,"completed_at":"2026-07-05T10:25:07.570Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 164.097839] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"894ffa4551b12ac4a425fc09","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-894ffa4551b12ac4a425fc09: start","job_status":"start","queue_duration_s":0.099004,"scheduling_latency_s":0.099024}1361gitlab # [ 164.099274] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"894ffa4551b12ac4a425fc09","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-894ffa4551b12ac4a425fc09: done: 0.019655 sec","job_status":"done","queue_duration_s":0.099004,"scheduling_latency_s":0.099024,"redis_calls":16,"redis_duration_s":0.003531,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001982,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000326019,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019655,"completed_at":"2026-07-05T10:25:07.591Z","load_balancing_strategy":"primary","db_duration_s":0.000398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 164.179450] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d19d786a0363c0c7c4b0f834","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d19d786a0363c0c7c4b0f834: start","job_status":"start","queue_duration_s":0.119921,"scheduling_latency_s":0.11994}1363gitlab # [ 164.181136] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d19d786a0363c0c7c4b0f834","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d19d786a0363c0c7c4b0f834: done: 0.01526 sec","job_status":"done","queue_duration_s":0.119921,"scheduling_latency_s":0.11994,"redis_calls":13,"redis_duration_s":0.002878,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01526,"completed_at":"2026-07-05T10:25:07.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 164.200315] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60eb3cc975ed0c5a2174d4c9","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60eb3cc975ed0c5a2174d4c9: start","job_status":"start","queue_duration_s":0.136475,"scheduling_latency_s":0.136503}1365gitlab # [ 164.278263] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60eb3cc975ed0c5a2174d4c9","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60eb3cc975ed0c5a2174d4c9: done: 0.015249 sec","job_status":"done","queue_duration_s":0.136475,"scheduling_latency_s":0.136503,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001872,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015249,"completed_at":"2026-07-05T10:25:07.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 164.281921] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3086dc5ba9e9a759964ee609","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3086dc5ba9e9a759964ee609: start","job_status":"start","queue_duration_s":0.152928,"scheduling_latency_s":0.152949}1367gitlab # [ 164.302517] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3086dc5ba9e9a759964ee609","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3086dc5ba9e9a759964ee609: done: 0.017036 sec","job_status":"done","queue_duration_s":0.152928,"scheduling_latency_s":0.152949,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017036,"completed_at":"2026-07-05T10:25:07.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 164.374170] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8d06361d124ff9b6d5c0f25","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8d06361d124ff9b6d5c0f25: start","job_status":"start","queue_duration_s":0.171213,"scheduling_latency_s":0.171234}1369gitlab # [ 164.447901] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8d06361d124ff9b6d5c0f25","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8d06361d124ff9b6d5c0f25: done: 0.014923 sec","job_status":"done","queue_duration_s":0.171213,"scheduling_latency_s":0.171234,"redis_calls":13,"redis_duration_s":0.0028450000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014923,"completed_at":"2026-07-05T10:25:07.658Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 164.450102] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3391a9a98d22107c457629e3","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3391a9a98d22107c457629e3: start","job_status":"start","queue_duration_s":0.187398,"scheduling_latency_s":0.187454}1371gitlab # [ 164.519900] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3391a9a98d22107c457629e3","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3391a9a98d22107c457629e3: done: 0.014766 sec","job_status":"done","queue_duration_s":0.187398,"scheduling_latency_s":0.187454,"redis_calls":13,"redis_duration_s":0.002823,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014766,"completed_at":"2026-07-05T10:25:07.674Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 164.527199] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c60036813f26fc7707fce7ea","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c60036813f26fc7707fce7ea: start","job_status":"start","queue_duration_s":0.203657,"scheduling_latency_s":0.203678}1373gitlab # [ 164.597144] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c60036813f26fc7707fce7ea","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c60036813f26fc7707fce7ea: done: 0.014842 sec","job_status":"done","queue_duration_s":0.203657,"scheduling_latency_s":0.203678,"redis_calls":13,"redis_duration_s":0.002786,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014842,"completed_at":"2026-07-05T10:25:07.691Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 164.668374] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b68a87533975f3e8edc7f745","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b68a87533975f3e8edc7f745: start","job_status":"start","queue_duration_s":0.219747,"scheduling_latency_s":0.219768}1375gitlab # [ 164.669981] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b68a87533975f3e8edc7f745","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b68a87533975f3e8edc7f745: done: 0.014528 sec","job_status":"done","queue_duration_s":0.219747,"scheduling_latency_s":0.219768,"redis_calls":13,"redis_duration_s":0.002904,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014528,"completed_at":"2026-07-05T10:25:07.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 164.689952] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97afb55cbffdd963ac5447b3","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97afb55cbffdd963ac5447b3: start","job_status":"start","queue_duration_s":0.23556,"scheduling_latency_s":0.235581}1377gitlab # [ 164.765071] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97afb55cbffdd963ac5447b3","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97afb55cbffdd963ac5447b3: done: 0.014601 sec","job_status":"done","queue_duration_s":0.23556,"scheduling_latency_s":0.235581,"redis_calls":13,"redis_duration_s":0.0027359999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001872,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014601,"completed_at":"2026-07-05T10:25:07.722Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 164.836151] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4fcb7b9c46e4b83aee44fed","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4fcb7b9c46e4b83aee44fed: start","job_status":"start","queue_duration_s":0.251706,"scheduling_latency_s":0.251726}1379gitlab # [ 164.837476] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4fcb7b9c46e4b83aee44fed","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4fcb7b9c46e4b83aee44fed: done: 0.014961 sec","job_status":"done","queue_duration_s":0.251706,"scheduling_latency_s":0.251726,"redis_calls":13,"redis_duration_s":0.003215,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014961,"completed_at":"2026-07-05T10:25:07.739Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 164.919908] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51b8755d4a348d3b5af66108","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51b8755d4a348d3b5af66108: start","job_status":"start","queue_duration_s":0.267892,"scheduling_latency_s":0.267913}1381gitlab # [ 164.921403] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51b8755d4a348d3b5af66108","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51b8755d4a348d3b5af66108: done: 0.014604 sec","job_status":"done","queue_duration_s":0.267892,"scheduling_latency_s":0.267913,"redis_calls":13,"redis_duration_s":0.002741,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014604,"completed_at":"2026-07-05T10:25:07.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 164.991921] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4dfa39287091bc2bf62ccc1","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4dfa39287091bc2bf62ccc1: start","job_status":"start","queue_duration_s":0.283737,"scheduling_latency_s":0.283758}1383gitlab # [ 164.993327] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4dfa39287091bc2bf62ccc1","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4dfa39287091bc2bf62ccc1: done: 0.02111 sec","job_status":"done","queue_duration_s":0.283737,"scheduling_latency_s":0.283758,"redis_calls":13,"redis_duration_s":0.006788000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004876,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02111,"completed_at":"2026-07-05T10:25:07.777Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 165.062136] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01e85101e3cd492adde2ad4d","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01e85101e3cd492adde2ad4d: start","job_status":"start","queue_duration_s":0.306252,"scheduling_latency_s":0.306278}1385gitlab # [ 165.063437] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01e85101e3cd492adde2ad4d","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01e85101e3cd492adde2ad4d: done: 0.015893 sec","job_status":"done","queue_duration_s":0.306252,"scheduling_latency_s":0.306278,"redis_calls":13,"redis_duration_s":0.002773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015893,"completed_at":"2026-07-05T10:25:07.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 165.142138] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c4e062cb2bd375f429d1bf4","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c4e062cb2bd375f429d1bf4: start","job_status":"start","queue_duration_s":0.323355,"scheduling_latency_s":0.323376}1387gitlab # [ 165.143467] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c4e062cb2bd375f429d1bf4","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c4e062cb2bd375f429d1bf4: done: 0.015537 sec","job_status":"done","queue_duration_s":0.323355,"scheduling_latency_s":0.323376,"redis_calls":13,"redis_duration_s":0.0035730000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002752,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015537,"completed_at":"2026-07-05T10:25:07.811Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 165.213360] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ac3453467c783e5f74209ad","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ac3453467c783e5f74209ad: start","job_status":"start","queue_duration_s":0.34011,"scheduling_latency_s":0.34013}1389gitlab # [ 165.214537] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ac3453467c783e5f74209ad","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ac3453467c783e5f74209ad: done: 0.014765 sec","job_status":"done","queue_duration_s":0.34011,"scheduling_latency_s":0.34013,"redis_calls":13,"redis_duration_s":0.002743,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014765,"completed_at":"2026-07-05T10:25:07.827Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 165.291396] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bedf89bb942f32f0ecf1fc78","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bedf89bb942f32f0ecf1fc78: start","job_status":"start","queue_duration_s":0.356099,"scheduling_latency_s":0.35612}1391gitlab # [ 165.292964] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bedf89bb942f32f0ecf1fc78","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bedf89bb942f32f0ecf1fc78: done: 0.014756 sec","job_status":"done","queue_duration_s":0.356099,"scheduling_latency_s":0.35612,"redis_calls":13,"redis_duration_s":0.0027559999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014756,"completed_at":"2026-07-05T10:25:07.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 165.379880] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7792a6c672b237b15a399bde","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7792a6c672b237b15a399bde: start","job_status":"start","queue_duration_s":0.372082,"scheduling_latency_s":0.372102}1393gitlab # [ 165.381101] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7792a6c672b237b15a399bde","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7792a6c672b237b15a399bde: done: 0.014733 sec","job_status":"done","queue_duration_s":0.372082,"scheduling_latency_s":0.372102,"redis_calls":13,"redis_duration_s":0.002766,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014733,"completed_at":"2026-07-05T10:25:07.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 165.422889] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e2e644d07d001ca1df86c99","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e2e644d07d001ca1df86c99: start","job_status":"start","queue_duration_s":0.388049,"scheduling_latency_s":0.38807}1395gitlab # [ 165.498386] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e2e644d07d001ca1df86c99","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e2e644d07d001ca1df86c99: done: 0.014611 sec","job_status":"done","queue_duration_s":0.388049,"scheduling_latency_s":0.38807,"redis_calls":13,"redis_duration_s":0.002764,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014611,"completed_at":"2026-07-05T10:25:07.875Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 165.504692] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adf2f24f6f66eb61882237da","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adf2f24f6f66eb61882237da: start","job_status":"start","queue_duration_s":0.404065,"scheduling_latency_s":0.404086}1397gitlab # [ 165.586643] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adf2f24f6f66eb61882237da","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adf2f24f6f66eb61882237da: done: 0.014859 sec","job_status":"done","queue_duration_s":0.404065,"scheduling_latency_s":0.404086,"redis_calls":13,"redis_duration_s":0.00295,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014859,"completed_at":"2026-07-05T10:25:07.891Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 165.588697] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"839bc271b12ee99f0ac937da","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-839bc271b12ee99f0ac937da: start","job_status":"start","queue_duration_s":0.420145,"scheduling_latency_s":0.420165}1399gitlab # [ 165.605936] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"839bc271b12ee99f0ac937da","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-839bc271b12ee99f0ac937da: done: 0.02148 sec","job_status":"done","queue_duration_s":0.420145,"scheduling_latency_s":0.420165,"redis_calls":13,"redis_duration_s":0.0043230000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001043,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00328,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02148,"completed_at":"2026-07-05T10:25:07.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 165.675952] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aa60abbba064321ef1011b7","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aa60abbba064321ef1011b7: start","job_status":"start","queue_duration_s":0.443147,"scheduling_latency_s":0.443174}1401gitlab # [ 165.749573] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aa60abbba064321ef1011b7","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aa60abbba064321ef1011b7: done: 0.015627 sec","job_status":"done","queue_duration_s":0.443147,"scheduling_latency_s":0.443174,"redis_calls":13,"redis_duration_s":0.0029189999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015627,"completed_at":"2026-07-05T10:25:07.931Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 165.751092] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65741524c17e3bd111546970","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65741524c17e3bd111546970: start","job_status":"start","queue_duration_s":0.460242,"scheduling_latency_s":0.460262}1403gitlab # [ 165.824437] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65741524c17e3bd111546970","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65741524c17e3bd111546970: done: 0.014932 sec","job_status":"done","queue_duration_s":0.460242,"scheduling_latency_s":0.460262,"redis_calls":13,"redis_duration_s":0.002831,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014932,"completed_at":"2026-07-05T10:25:07.947Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 165.829950] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"074670e8ec33f6efc9940492","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-074670e8ec33f6efc9940492: start","job_status":"start","queue_duration_s":0.476465,"scheduling_latency_s":0.476493}1405gitlab # [ 165.850163] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"074670e8ec33f6efc9940492","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-074670e8ec33f6efc9940492: done: 0.015284 sec","job_status":"done","queue_duration_s":0.476465,"scheduling_latency_s":0.476493,"redis_calls":13,"redis_duration_s":0.002836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015284,"completed_at":"2026-07-05T10:25:07.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 165.932273] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed866027af683abfc54f1f7d","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed866027af683abfc54f1f7d: start","job_status":"start","queue_duration_s":0.492957,"scheduling_latency_s":0.492978}1407gitlab # [ 165.933892] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed866027af683abfc54f1f7d","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed866027af683abfc54f1f7d: done: 0.014989 sec","job_status":"done","queue_duration_s":0.492957,"scheduling_latency_s":0.492978,"redis_calls":13,"redis_duration_s":0.0032010000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002282,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014989,"completed_at":"2026-07-05T10:25:07.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 166.010414] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbe739ea426c7617340ab8d1","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbe739ea426c7617340ab8d1: start","job_status":"start","queue_duration_s":0.509472,"scheduling_latency_s":0.509501}1409gitlab # [ 166.012408] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbe739ea426c7617340ab8d1","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbe739ea426c7617340ab8d1: done: 0.014988 sec","job_status":"done","queue_duration_s":0.509472,"scheduling_latency_s":0.509501,"redis_calls":13,"redis_duration_s":0.002916,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014988,"completed_at":"2026-07-05T10:25:07.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 166.106268] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:07.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebc3b90fd07373ff1013413b","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebc3b90fd07373ff1013413b: start","job_status":"start","queue_duration_s":0.525913,"scheduling_latency_s":0.525934}1411gitlab # [ 166.107886] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:08.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebc3b90fd07373ff1013413b","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebc3b90fd07373ff1013413b: done: 0.014845 sec","job_status":"done","queue_duration_s":0.525913,"scheduling_latency_s":0.525934,"redis_calls":13,"redis_duration_s":0.0029179999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014845,"completed_at":"2026-07-05T10:25:08.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 166.215523] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:08.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"018304bf28413c545d928275","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-018304bf28413c545d928275: start","job_status":"start","queue_duration_s":0.541994,"scheduling_latency_s":0.542015}1413gitlab # [ 166.217180] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:08.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"018304bf28413c545d928275","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-018304bf28413c545d928275: done: 0.014847 sec","job_status":"done","queue_duration_s":0.541994,"scheduling_latency_s":0.542015,"redis_calls":13,"redis_duration_s":0.0028469999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014847,"completed_at":"2026-07-05T10:25:08.029Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 166.299527] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:08.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51963bc2af163a11f3bba6f4","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51963bc2af163a11f3bba6f4: start","job_status":"start","queue_duration_s":0.558088,"scheduling_latency_s":0.558109}1415gitlab # [ 166.300944] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:08.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51963bc2af163a11f3bba6f4","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51963bc2af163a11f3bba6f4: done: 0.01474 sec","job_status":"done","queue_duration_s":0.558088,"scheduling_latency_s":0.558109,"redis_calls":13,"redis_duration_s":0.002803,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01474,"completed_at":"2026-07-05T10:25:08.045Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 166.319258] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:08.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b62d26ffd0f8bec6ee0c60dc","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b62d26ffd0f8bec6ee0c60dc: start","job_status":"start","queue_duration_s":0.57421,"scheduling_latency_s":0.574232}1417gitlab # [ 166.672443] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2125671923 wd_nsec: 21256707771418gitlab # [ 166.427849] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b62d26ffd0f8bec6ee0c60dc","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b62d26ffd0f8bec6ee0c60dc: done: 2.257806 sec","job_status":"done","queue_duration_s":0.57421,"scheduling_latency_s":0.574232,"redis_calls":13,"redis_duration_s":0.0036769999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":2.257806,"completed_at":"2026-07-05T10:25:10.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 166.430907] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7ae7d36ddd52bdd69479d95","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7ae7d36ddd52bdd69479d95: start","job_status":"start","queue_duration_s":2.835843,"scheduling_latency_s":2.835897}1420gitlab # [ 166.465599] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7ae7d36ddd52bdd69479d95","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7ae7d36ddd52bdd69479d95: done: 0.034537 sec","job_status":"done","queue_duration_s":2.835843,"scheduling_latency_s":2.835897,"redis_calls":13,"redis_duration_s":0.004438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001269,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034537,"completed_at":"2026-07-05T10:25:10.343Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 166.560525] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"487897aff2b6baf3f25e19ab","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-487897aff2b6baf3f25e19ab: start","job_status":"start","queue_duration_s":2.873035,"scheduling_latency_s":2.873075}1422gitlab # [ 166.561978] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"487897aff2b6baf3f25e19ab","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-487897aff2b6baf3f25e19ab: done: 0.017167 sec","job_status":"done","queue_duration_s":2.873035,"scheduling_latency_s":2.873075,"redis_calls":13,"redis_duration_s":0.002957,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017167,"completed_at":"2026-07-05T10:25:10.362Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 166.635070] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"669a66c2011ef647f405a027","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-669a66c2011ef647f405a027: start","job_status":"start","queue_duration_s":2.891551,"scheduling_latency_s":2.891572}1424gitlab # [ 166.636396] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"669a66c2011ef647f405a027","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-669a66c2011ef647f405a027: done: 0.02038 sec","job_status":"done","queue_duration_s":2.891551,"scheduling_latency_s":2.891572,"redis_calls":13,"redis_duration_s":0.003647,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002359,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02038,"completed_at":"2026-07-05T10:25:10.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 166.714515] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfd04ba748375e0c4edebb4c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfd04ba748375e0c4edebb4c: start","job_status":"start","queue_duration_s":2.913203,"scheduling_latency_s":2.913225}1426gitlab # [ 166.715892] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfd04ba748375e0c4edebb4c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfd04ba748375e0c4edebb4c: done: 0.015079 sec","job_status":"done","queue_duration_s":2.913203,"scheduling_latency_s":2.913225,"redis_calls":13,"redis_duration_s":0.002874,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015079,"completed_at":"2026-07-05T10:25:10.401Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 166.786068] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"851431eda2aeb328486ea1dc","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-851431eda2aeb328486ea1dc: start","job_status":"start","queue_duration_s":2.929572,"scheduling_latency_s":2.929593}1428gitlab # [ 166.787265] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"851431eda2aeb328486ea1dc","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-851431eda2aeb328486ea1dc: done: 0.014718 sec","job_status":"done","queue_duration_s":2.929572,"scheduling_latency_s":2.929593,"redis_calls":13,"redis_duration_s":0.002728,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014718,"completed_at":"2026-07-05T10:25:10.417Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 166.865283] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d7fbc3bdf51b472309df274","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d7fbc3bdf51b472309df274: start","job_status":"start","queue_duration_s":2.945567,"scheduling_latency_s":2.945587}1430gitlab # [ 166.866172] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d7fbc3bdf51b472309df274","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d7fbc3bdf51b472309df274: done: 0.01517 sec","job_status":"done","queue_duration_s":2.945567,"scheduling_latency_s":2.945587,"redis_calls":13,"redis_duration_s":0.002941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01517,"completed_at":"2026-07-05T10:25:10.433Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 166.912403] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e9ded4e45b2e755a3aad93a","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e9ded4e45b2e755a3aad93a: start","job_status":"start","queue_duration_s":2.962005,"scheduling_latency_s":2.962027}1432gitlab # [ 166.981326] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e9ded4e45b2e755a3aad93a","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e9ded4e45b2e755a3aad93a: done: 0.015296 sec","job_status":"done","queue_duration_s":2.962005,"scheduling_latency_s":2.962027,"redis_calls":13,"redis_duration_s":0.0031729999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015296,"completed_at":"2026-07-05T10:25:10.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 167.050484] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30aa6bfdbc0fa3076068581c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30aa6bfdbc0fa3076068581c: start","job_status":"start","queue_duration_s":2.97868,"scheduling_latency_s":2.9787}1434gitlab # [ 167.052152] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30aa6bfdbc0fa3076068581c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30aa6bfdbc0fa3076068581c: done: 0.017382 sec","job_status":"done","queue_duration_s":2.97868,"scheduling_latency_s":2.9787,"redis_calls":13,"redis_duration_s":0.003125,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017382,"completed_at":"2026-07-05T10:25:10.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 167.125111] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5de6092f01f09e28d7bb594c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5de6092f01f09e28d7bb594c: start","job_status":"start","queue_duration_s":2.997641,"scheduling_latency_s":2.997669}1436gitlab # [ 167.126502] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5de6092f01f09e28d7bb594c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5de6092f01f09e28d7bb594c: done: 0.015938 sec","job_status":"done","queue_duration_s":2.997641,"scheduling_latency_s":2.997669,"redis_calls":13,"redis_duration_s":0.002798,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015938,"completed_at":"2026-07-05T10:25:10.486Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 167.198419] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f769c302234256de839f980","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f769c302234256de839f980: start","job_status":"start","queue_duration_s":3.014833,"scheduling_latency_s":3.014853}1438gitlab # [ 167.200004] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f769c302234256de839f980","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f769c302234256de839f980: done: 0.015073 sec","job_status":"done","queue_duration_s":3.014833,"scheduling_latency_s":3.014853,"redis_calls":13,"redis_duration_s":0.00304,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015073,"completed_at":"2026-07-05T10:25:10.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 167.277289] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5038fd155fe0534e6a825941","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5038fd155fe0534e6a825941: start","job_status":"start","queue_duration_s":3.031157,"scheduling_latency_s":3.031178}1440gitlab # [ 167.278852] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5038fd155fe0534e6a825941","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5038fd155fe0534e6a825941: done: 0.015432 sec","job_status":"done","queue_duration_s":3.031157,"scheduling_latency_s":3.031178,"redis_calls":13,"redis_duration_s":0.0031279999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015432,"completed_at":"2026-07-05T10:25:10.519Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 167.353228] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba15e5bea70800dbf84b9c6c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba15e5bea70800dbf84b9c6c: start","job_status":"start","queue_duration_s":3.047996,"scheduling_latency_s":3.048017}1442gitlab # [ 167.355143] setup-gitlab-start[2683]: W, [2026-07-05T10:25:11.094871 #2683] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1443gitlab # [ 167.355907] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba15e5bea70800dbf84b9c6c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba15e5bea70800dbf84b9c6c: done: 0.014833 sec","job_status":"done","queue_duration_s":3.047996,"scheduling_latency_s":3.048017,"redis_calls":13,"redis_duration_s":0.002818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014833,"completed_at":"2026-07-05T10:25:10.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 167.449722] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f84c799d5e659c7e4521a0c4","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f84c799d5e659c7e4521a0c4: start","job_status":"start","queue_duration_s":3.064087,"scheduling_latency_s":3.064109}1445gitlab # [ 167.451210] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f84c799d5e659c7e4521a0c4","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f84c799d5e659c7e4521a0c4: done: 0.01485 sec","job_status":"done","queue_duration_s":3.064087,"scheduling_latency_s":3.064109,"redis_calls":13,"redis_duration_s":0.002757,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01485,"completed_at":"2026-07-05T10:25:10.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 167.527343] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8376f2f7a33076e84a0ab4c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8376f2f7a33076e84a0ab4c: start","job_status":"start","queue_duration_s":3.080166,"scheduling_latency_s":3.080187}1447gitlab # [ 167.528984] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8376f2f7a33076e84a0ab4c","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8376f2f7a33076e84a0ab4c: done: 0.014829 sec","job_status":"done","queue_duration_s":3.080166,"scheduling_latency_s":3.080187,"redis_calls":13,"redis_duration_s":0.002725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014829,"completed_at":"2026-07-05T10:25:10.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 167.603458] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75d31218c9c958f8cc59d5d7","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75d31218c9c958f8cc59d5d7: start","job_status":"start","queue_duration_s":3.096237,"scheduling_latency_s":3.096258}1449gitlab # [ 167.604755] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75d31218c9c958f8cc59d5d7","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75d31218c9c958f8cc59d5d7: done: 0.014718 sec","job_status":"done","queue_duration_s":3.096237,"scheduling_latency_s":3.096258,"redis_calls":13,"redis_duration_s":0.002726,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014718,"completed_at":"2026-07-05T10:25:10.583Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 167.642799] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d452118e6cdd30dfb71c874","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d452118e6cdd30dfb71c874: start","job_status":"start","queue_duration_s":3.112211,"scheduling_latency_s":3.112232}1451gitlab # [ 167.663639] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d452118e6cdd30dfb71c874","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d452118e6cdd30dfb71c874: done: 0.015115 sec","job_status":"done","queue_duration_s":3.112211,"scheduling_latency_s":3.112232,"redis_calls":13,"redis_duration_s":0.003061,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015115,"completed_at":"2026-07-05T10:25:10.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 167.779302] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d2a968efc7153d6550c94d9","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d2a968efc7153d6550c94d9: start","job_status":"start","queue_duration_s":3.128642,"scheduling_latency_s":3.128663}1453gitlab # [ 167.780917] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d2a968efc7153d6550c94d9","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d2a968efc7153d6550c94d9: done: 0.014736 sec","job_status":"done","queue_duration_s":3.128642,"scheduling_latency_s":3.128663,"redis_calls":13,"redis_duration_s":0.003032,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001069,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014736,"completed_at":"2026-07-05T10:25:10.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 167.870120] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff2553223f53e98f467ec82b","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff2553223f53e98f467ec82b: start","job_status":"start","queue_duration_s":3.144804,"scheduling_latency_s":3.144824}1455gitlab # [ 167.871498] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff2553223f53e98f467ec82b","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff2553223f53e98f467ec82b: done: 0.017246 sec","job_status":"done","queue_duration_s":3.144804,"scheduling_latency_s":3.144824,"redis_calls":13,"redis_duration_s":0.002951,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017246,"completed_at":"2026-07-05T10:25:10.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 167.946308] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8085b7b6c26a193d807062e","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8085b7b6c26a193d807062e: start","job_status":"start","queue_duration_s":3.163528,"scheduling_latency_s":3.163554}1457gitlab # [ 167.947936] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8085b7b6c26a193d807062e","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8085b7b6c26a193d807062e: done: 0.015108 sec","job_status":"done","queue_duration_s":3.163528,"scheduling_latency_s":3.163554,"redis_calls":13,"redis_duration_s":0.002787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015108,"completed_at":"2026-07-05T10:25:10.651Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 168.033132] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5898facc8fab6da59db305f9","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5898facc8fab6da59db305f9: start","job_status":"start","queue_duration_s":3.17985,"scheduling_latency_s":3.17987}1459gitlab # [ 168.034554] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5898facc8fab6da59db305f9","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5898facc8fab6da59db305f9: done: 0.014757 sec","job_status":"done","queue_duration_s":3.17985,"scheduling_latency_s":3.17987,"redis_calls":13,"redis_duration_s":0.0028640000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014757,"completed_at":"2026-07-05T10:25:10.667Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 168.054076] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5efd005acd9fd642c2c586ac","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5efd005acd9fd642c2c586ac: start","job_status":"start","queue_duration_s":3.195938,"scheduling_latency_s":3.195959}1461gitlab # [ 168.123515] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:10.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5efd005acd9fd642c2c586ac","created_at":"2026-07-05T10:25:07.407Z","trace_propagation_headers":{"sentry-trace":"85a1f89d867f49919f154cc87475bd5d-5e6e339071234dbf","baggage":"sentry-trace_id=85a1f89d867f49919f154cc87475bd5d,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"716de349d69f273e762597236597f146","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:07.472Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5efd005acd9fd642c2c586ac: done: 0.014868 sec","job_status":"done","queue_duration_s":3.195938,"scheduling_latency_s":3.195959,"redis_calls":13,"redis_duration_s":0.002899,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014868,"completed_at":"2026-07-05T10:25:10.683Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 189.532436] runuser[2682]: pam_unix(runuser:session): session closed for user gitlab1463gitlab # [ 189.536794] setup-gitlab-start[2906]: ++ jq -r .token1464gitlab # [ 189.541790] setup-gitlab-start[2907]: +++ date -d '+7 days' +%F1465gitlab # [ 189.546396] setup-gitlab-start[2905]: ++ curl -fs -X POST -H 'Authorization: Bearer 11647bc5113bc66e4ef49e8748079ed2ee411c36e9b1e0152ef635dc9b1218e9' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-12"}' http://gitlab/api/v4/users/1/personal_access_tokens1466gitlab # [ 189.780889] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:33.657Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"144b5cade9858424ba8e85d9","created_at":"2026-07-05T10:25:33.602Z","trace_propagation_headers":{"sentry-trace":"4b6004f287894231b86ea11fdeacd65b-9e9803d180354372","baggage":"sentry-trace_id=4b6004f287894231b86ea11fdeacd65b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZPR3J535M4A5MHJ6M1PW","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:33a43b88deca7a65468c683c657ddf0e6f689675ea83e41780108b0e80817df3","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:33.652Z","job_size_bytes":451,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-144b5cade9858424ba8e85d9: start","job_status":"start","queue_duration_s":0.005102,"scheduling_latency_s":0.005141}1467gitlab # [ 189.839480] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:33 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 2211468gitlab # [ 189.846144] setup-gitlab-start[2640]: + TOKEN=glpat-FK1H81Lts7-5dprLmWEaFW86MQp1OjEH.01.0w1jfmbai1469gitlab # [ 189.848216] setup-gitlab-start[2640]: + test -n glpat-FK1H81Lts7-5dprLmWEaFW86MQp1OjEH.01.0w1jfmbai1470gitlab # [ 189.850275] setup-gitlab-start[2640]: + test glpat-FK1H81Lts7-5dprLmWEaFW86MQp1OjEH.01.0w1jfmbai '!=' null1471gitlab # [ 189.852417] setup-gitlab-start[2640]: + mkdir -p /var/lib/secrets1472gitlab # [ 189.853902] setup-gitlab-start[2640]: + echo glpat-FK1H81Lts7-5dprLmWEaFW86MQp1OjEH.01.0w1jfmbai1473gitlab # [ 189.855941] setup-gitlab-start[2640]: + chmod 644 /var/lib/secrets/gitlab-token1474gitlab # [ 189.857698] setup-gitlab-start[2640]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-FK1H81Lts7-5dprLmWEaFW86MQp1OjEH.01.0w1jfmbai' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1475gitlab # [ 189.932644] sidekiq-cluster[2102]: {"severity":"WARN","time":"2026-07-05T10:25:33.809Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"144b5cade9858424ba8e85d9","created_at":"2026-07-05T10:25:33.602Z","trace_propagation_headers":{"sentry-trace":"4b6004f287894231b86ea11fdeacd65b-9e9803d180354372","baggage":"sentry-trace_id=4b6004f287894231b86ea11fdeacd65b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZPR3J535M4A5MHJ6M1PW","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:33a43b88deca7a65468c683c657ddf0e6f689675ea83e41780108b0e80817df3","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:33.652Z","job_size_bytes":451,"pid":2102,"sidekiq_tid":"lyo6","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-144b5cade9858424ba8e85d9: fail: 0.151573 sec","job_status":"fail","queue_duration_s":0.005102,"scheduling_latency_s":0.005141,"redis_calls":11,"redis_duration_s":0.004281,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001131,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000711,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000529397,"duration_s":0.151573,"completed_at":"2026-07-05T10:25:33.809Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.054842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 190.865352] gitaly[1389]: time="2026-07-05T10:25:34.742Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:44.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:34.741" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1477gitlab # [ 190.868300] gitaly[1389]: time="2026-07-05T10:25:34.742Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:44.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:34.741" grpc.time_ms=0.758 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1478gitlab # [ 191.059634] gitaly[1389]: time="2026-07-05T10:25:34.937Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:44.206" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:34.936" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1479gitlab # [ 191.102904] gitaly[1389]: time="2026-07-05T10:25:34.937Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:44.206" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:34.936" grpc.time_ms=0.661 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1480gitlab # [ 191.132445] gitaly[1389]: time="2026-07-05T10:25:34.983Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT diskcache=e1c6e39b-7e5c-4d94-8ff7-c34478ae3c62 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-05T10:25:44.213" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:34.944" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1481gitlab # [ 191.149506] gitaly[1389]: time="2026-07-05T10:25:34.984Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=32 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-05T10:25:44.213" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:34.944" grpc.time_ms=39.946 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1482gitlab # [ 191.171839] gitaly[1389]: time="2026-07-05T10:25:34.997Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:44.265" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:34.996" pid=1389 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1483gitlab # [ 191.189237] gitaly[1389]: time="2026-07-05T10:25:35.010Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:44.265" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:34.996" grpc.time_ms=13.853 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1484gitlab # [ 191.207893] gitaly[1389]: time="2026-07-05T10:25:35.028Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-05T10:25:45.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.018" grpc.time_ms=10.143 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1485gitlab # [ 191.210836] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.060Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3221f73db22079f5799461a4","created_at":"2026-07-05T10:25:35.041Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.058Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3221f73db22079f5799461a4: start","job_status":"start","queue_duration_s":0.001967,"scheduling_latency_s":0.002201}1486gitlab # [ 191.212814] gitaly[1389]: time="2026-07-05T10:25:35.079Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:45.067" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.078" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1487gitlab # [ 191.214208] gitaly[1389]: time="2026-07-05T10:25:35.079Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:45.067" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.078" grpc.time_ms=0.6 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1488gitlab # [ 191.240558] gitaly[1389]: time="2026-07-05T10:25:35.118Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT diskcache=49389f5e-9e96-4a31-804b-e9fda0c9f2f3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-05T10:25:45.074" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.084" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1489gitlab # [ 191.242446] gitaly[1389]: time="2026-07-05T10:25:35.118Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=326 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-05T10:25:45.074" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.084" grpc.time_ms=34.114 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1490gitlab # [ 191.323360] gitaly[1389]: time="2026-07-05T10:25:35.126Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:45.115" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.125" pid=1389 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1491gitlab # [ 191.324039] gitaly[1389]: time="2026-07-05T10:25:35.127Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:45.115" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.125" grpc.time_ms=0.402 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1492gitlab # [ 191.324949] gitaly[1389]: time="2026-07-05T10:25:35.139Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-05T10:25:45.119" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.129" grpc.time_ms=10.373 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1493gitlab # [ 191.326234] gitaly[1389]: time="2026-07-05T10:25:35.163Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:45.152" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.163" grpc.time_ms=0.232 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1494gitlab # [ 191.333982] gitaly[1389]: time="2026-07-05T10:25:35.168Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-05T10:25:45.157" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.168" grpc.time_ms=0.299 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1495gitlab # [ 191.456481] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.333Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3221f73db22079f5799461a4","created_at":"2026-07-05T10:25:35.041Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.058Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3221f73db22079f5799461a4: done: 0.273933 sec","job_status":"done","queue_duration_s":0.001967,"scheduling_latency_s":0.002201,"redis_calls":10,"redis_duration_s":0.004828,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002575,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.222,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.22,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.239448589,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.273933,"completed_at":"2026-07-05T10:25:35.333Z","load_balancing_strategy":"primary","db_duration_s":0.221931,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1496gitlab # [ 191.527456] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.336Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"448a1b45b83b3fcb761eb3d4","created_at":"2026-07-05T10:25:35.065Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.072Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-448a1b45b83b3fcb761eb3d4: start","job_status":"start","queue_duration_s":0.263929,"scheduling_latency_s":0.263958}1497gitlab # [ 191.529098] gitaly[1389]: time="2026-07-05T10:25:35.385Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZQ1DFT13D0YCTS8BSSQT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-05T10:25:45.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:35.377" grpc.time_ms=8.221 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1498gitlab # [ 191.586698] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.464Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"448a1b45b83b3fcb761eb3d4","created_at":"2026-07-05T10:25:35.065Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.072Z","job_size_bytes":2,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-448a1b45b83b3fcb761eb3d4: done: 0.128599 sec","job_status":"done","queue_duration_s":0.263929,"scheduling_latency_s":0.263958,"redis_calls":14,"redis_duration_s":0.004229,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000678,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002344,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09919054,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.128599,"completed_at":"2026-07-05T10:25:35.464Z","load_balancing_strategy":"primary","db_duration_s":0.06144,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1499gitlab # [ 191.589533] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.465Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c9e0773ee2c840ee4b5f63ec","created_at":"2026-07-05T10:25:35.319Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.325Z","job_size_bytes":5,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c9e0773ee2c840ee4b5f63ec: start","job_status":"start","queue_duration_s":0.140469,"scheduling_latency_s":0.140491}1500gitlab # [ 191.675409] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.551Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c9e0773ee2c840ee4b5f63ec","created_at":"2026-07-05T10:25:35.319Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.325Z","job_size_bytes":5,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c9e0773ee2c840ee4b5f63ec: done: 0.085902 sec","job_status":"done","queue_duration_s":0.140469,"scheduling_latency_s":0.140491,"redis_calls":12,"redis_duration_s":0.003512,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000655,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002304,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000514032,"duration_s":0.085902,"completed_at":"2026-07-05T10:25:35.551Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024453,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1501gitlab # [ 191.752313] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.554Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"8397bab61a0dc3e1dfe01fa6","created_at":"2026-07-05T10:25:35.354Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.358Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-8397bab61a0dc3e1dfe01fa6: start","job_status":"start","queue_duration_s":0.196274,"scheduling_latency_s":0.196329}1502gitlab # [ 191.952527] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:35 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 20861503gitlab # [ 191.961800] setup-gitlab-start[2640]: + mkdir -p /tmp/test-flake1504gitlab # [ 191.965840] setup-gitlab-start[2640]: + cd /tmp/test-flake1505gitlab # [ 191.969418] setup-gitlab-start[2640]: + git init -b master1506gitlab # [ 191.988989] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.866Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"8397bab61a0dc3e1dfe01fa6","created_at":"2026-07-05T10:25:35.354Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.358Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-8397bab61a0dc3e1dfe01fa6: done: 0.312072 sec","job_status":"done","queue_duration_s":0.196274,"scheduling_latency_s":0.196329,"redis_calls":11,"redis_duration_s":0.003225,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000776,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.276,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.276,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.113141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000360381,"duration_s":0.312072,"completed_at":"2026-07-05T10:25:35.866Z","load_balancing_strategy":"primary","db_duration_s":0.201396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 191.992792] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.870Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"086d12cea7fd4ee84dd08008","created_at":"2026-07-05T10:25:35.361Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.364Z","job_size_bytes":89,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-086d12cea7fd4ee84dd08008: start","job_status":"start","queue_duration_s":0.505244,"scheduling_latency_s":0.5053}1508gitlab # [ 192.082217] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:35.923Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"086d12cea7fd4ee84dd08008","created_at":"2026-07-05T10:25:35.361Z","trace_propagation_headers":{"sentry-trace":"48e2d354c8c24ec3b0e4557adbbad02d-4b2236d6ea8f4b0b","baggage":"sentry-trace_id=48e2d354c8c24ec3b0e4557adbbad02d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZQ1DFT13D0YCTS8BSSQT","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:35.364Z","job_size_bytes":89,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-086d12cea7fd4ee84dd08008: done: 0.053208 sec","job_status":"done","queue_duration_s":0.505244,"scheduling_latency_s":0.5053,"redis_calls":11,"redis_duration_s":0.00247,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000636,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001391,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000391669,"duration_s":0.053208,"completed_at":"2026-07-05T10:25:35.923Z","load_balancing_strategy":"primary","db_duration_s":0.014094,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 192.180138] setup-gitlab-start[2947]: Initialized empty Git repository in /tmp/test-flake/.git/1510gitlab # [ 192.182433] setup-gitlab-start[2640]: + git config user.name test1511gitlab # [ 192.184338] setup-gitlab-start[2640]: + git config user.email test@example.com1512gitlab # [ 192.187121] setup-gitlab-start[2640]: + cat1513gitlab # [ 192.192217] setup-gitlab-start[2640]: + git add flake.nix1514gitlab # [ 192.196352] setup-gitlab-start[2640]: + git commit -m 'initial commit'1515gitlab # [ 192.213414] setup-gitlab-start[2952]: [master (root-commit) 06bf5cb] initial commit1516gitlab # [ 192.215225] setup-gitlab-start[2952]: 1 file changed, 10 insertions(+)1517gitlab # [ 192.217223] setup-gitlab-start[2952]: create mode 100644 flake.nix1518gitlab # [ 192.219130] setup-gitlab-start[2640]: + git remote add origin http://oauth2:glpat-FK1H81Lts7-5dprLmWEaFW86MQp1OjEH.01.0w1jfmbai@gitlab/root/test-flake.git1519gitlab # [ 192.226497] setup-gitlab-start[2640]: + git push -u origin master1520gitlab # [ 192.310229] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:36 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 471521gitlab # [ 192.419352] gitaly[1389]: time="2026-07-05T10:25:36.296Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSDZMXP3JKH6NJFPPVZJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:36.278" grpc.time_ms=18.015 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1522gitlab # [ 192.442567] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:36 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1281523gitlab # [ 192.633725] gitaly[1389]: time="2026-07-05T10:25:36.511Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-05T10:25:46.134" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:36.504" grpc.time_ms=6.754 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1524gitlab # [ 192.685213] gitaly[1389]: time="2026-07-05T10:25:36.562Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-05T10:26:06.053" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-05T10:25:36.554" grpc.time_ms=7.621 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1525gitlab # [ 192.709834] gitaly[1389]: time="2026-07-05T10:25:36.587Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 duration_ms=139ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/allowed"1526gitlab # [ 192.814058] gitaly[1389]: time="2026-07-05T10:25:36.692Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 duration_ms=103ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/pre_receive"1527gitlab # [ 192.830093] gitaly[1389]: time="2026-07-05T10:25:36.708Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:36.447" grpc.time_ms=256.572 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1528gitlab # [ 192.896342] gitaly[1389]: time="2026-07-05T10:25:36.774Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:36.773" grpc.time_ms=0.542 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1529gitlab # [ 192.941789] gitaly[1389]: time="2026-07-05T10:25:36.819Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:36.817" grpc.time_ms=1.397 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1530gitlab # [ 192.982318] gitaly[1389]: time="2026-07-05T10:25:36.860Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:36.859" grpc.time_ms=0.375 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1531gitlab # [ 193.103380] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:36.980Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 06bf5cb9c56b0e286a64490181153eddf73461ec refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"c5f33f5b34c030f7afa69bce","created_at":"2026-07-05T10:25:36.934Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-2c2ba6371a2f4b7f","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:36.954Z","job_size_bytes":185,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-c5f33f5b34c030f7afa69bce: start","job_status":"start","queue_duration_s":0.025174,"scheduling_latency_s":0.025231}1532gitlab # [ 193.119398] gitaly[1389]: time="2026-07-05T10:25:36.997Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-05T10:25:46.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:36.990" grpc.time_ms=6.866 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1533gitlab # [ 193.196229] gitaly[1389]: time="2026-07-05T10:25:37.073Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1568 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-05T10:25:47.006" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:37.057" grpc.time_ms=16.281 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1534gitlab # [ 193.223871] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:37.101Z","message":"changes: \"0000000000000000000000000000000000000000 06bf5cb9c56b0e286a64490181153eddf73461ec refs/heads/master\\n\""}1535gitlab # [ 193.241457] gitaly[1389]: time="2026-07-05T10:25:37.119Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 duration_ms=216ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/post_receive"1536gitlab # [ 193.242848] gitaly[1389]: time="2026-07-05T10:25:37.120Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:36.902" grpc.time_ms=217.686 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1537gitlab # [ 193.275131] gitaly[1389]: time="2026-07-05T10:25:37.153Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 diskcache=4b35b184-0ce1-4e7b-ae89-322e3fd608c1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:36.394" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1538gitlab # [ 193.277336] gitaly[1389]: time="2026-07-05T10:25:37.154Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=156 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13118 command.oublock=32 command.real_time_ms=749 command.spawn_token_fork_ms=0 command.system_time_ms=112 command.user_time_ms=44 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:36.394" grpc.time_ms=760.374 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1539gitlab # [ 193.314627] gitlab-workhorse[1292]: time="2026-07-05T10:25:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521540gitlab # [ 193.318070] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:37 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 8601541gitlab # [ 193.321994] setup-gitlab-start[2956]: To http://gitlab/root/test-flake.git1542gitlab # [ 193.323195] setup-gitlab-start[2956]: * [new branch] master -> master1543gitlab # [ 193.328364] setup-gitlab-start[2956]: branch 'master' set up to track 'origin/master'.1544gitlab # [ 193.332642] systemd[1]: Finished setup-gitlab.service.1545gitlab # [ 193.338263] systemd[1]: Starting nixbot CI service...1546gitlab # [ 193.360704] gitaly[1389]: time="2026-07-05T10:25:37.238Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:47.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:37.231" grpc.time_ms=7.119 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1547gitlab # [ 193.442561] gitaly[1389]: time="2026-07-05T10:25:37.320Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-05T10:25:47.078" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:37.309" grpc.time_ms=10.452 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1548gitlab: (finished: waiting for unit setup-gitlab.service, in 194.43 seconds)1549gitlab: waiting for unit nixbot.service1550gitlab # [ 193.540654] gitaly[1389]: time="2026-07-05T10:25:37.418Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-05T10:25:47.179" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:37.410" grpc.time_ms=8.453 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1551gitlab # [ 193.614435] gitaly[1389]: time="2026-07-05T10:25:37.492Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:37.491" grpc.time_ms=0.445 pid=1389 span.kind=server system=grpc1552gitlab # [ 193.652739] gitaly[1389]: time="2026-07-05T10:25:37.530Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:37.529" grpc.time_ms=0.908 pid=1389 span.kind=server system=grpc1553gitlab # [ 193.678850] gitaly[1389]: time="2026-07-05T10:25:37.556Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 diskcache=8274cd3e-1b46-45f1-a6df-9c7c9c3cc536 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-05T10:25:47.229" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:37.460" pid=1389 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1554gitlab # [ 193.680864] gitaly[1389]: time="2026-07-05T10:25:37.558Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=58 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4667 command.oublock=8 command.real_time_ms=89 command.spawn_token_fork_ms=1 command.system_time_ms=42 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-05T10:25:47.229" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:37.460" grpc.time_ms=96.491 pid=1389 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1555gitlab # [ 193.727977] gitaly[1389]: time="2026-07-05T10:25:37.606Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-05T10:25:47.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:37.605" grpc.time_ms=0.171 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1556gitlab # [ 193.765619] gitaly[1389]: time="2026-07-05T10:25:37.643Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=942 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-05T10:25:47.032" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:37.632" grpc.time_ms=10.59 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1557gitlab # [ 193.857979] gitaly[1389]: time="2026-07-05T10:25:37.736Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-05T10:26:07.030" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:37.730" grpc.time_ms=5.37 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1558gitlab # [ 193.891130] gitaly[1389]: time="2026-07-05T10:25:37.769Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-05T10:25:47.168" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:37.768" grpc.time_ms=0.353 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1559gitlab # [ 193.917707] gitaly[1389]: time="2026-07-05T10:25:37.795Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-05T10:25:47.190" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-05T10:25:37.791" grpc.time_ms=4.294 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1560gitlab # [ 193.977662] gitaly[1389]: time="2026-07-05T10:25:37.855Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:47.248" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:37.848" grpc.time_ms=6.92 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1561gitlab # [ 194.007479] gitaly[1389]: time="2026-07-05T10:25:37.885Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:47.278" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:37.878" grpc.time_ms=6.581 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1562gitlab # [ 194.064428] gitaly[1389]: time="2026-07-05T10:25:37.942Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-05T10:26:07.231" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:37.931" grpc.time_ms=10.026 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1563gitlab # [ 194.400703] nixbot[3043]: {"ts": "2026-07-05T10:25:38.278Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1564gitlab # [ 194.433173] nixbot[3043]: {"ts": "2026-07-05T10:25:38.311Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1565gitlab # [ 194.452552] nixbot[3043]: {"ts": "2026-07-05T10:25:38.330Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1566gitlab # [ 194.456381] nixbot[3043]: {"ts": "2026-07-05T10:25:38.334Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1567gitlab # [ 194.460299] nixbot[3043]: {"ts": "2026-07-05T10:25:38.337Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1568gitlab # [ 194.464269] nixbot[3043]: {"ts": "2026-07-05T10:25:38.342Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1569gitlab # [ 194.468167] nixbot[3043]: {"ts": "2026-07-05T10:25:38.345Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1570gitlab # [ 194.472138] nixbot[3043]: {"ts": "2026-07-05T10:25:38.347Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1571gitlab # [ 194.475373] nixbot[3043]: {"ts": "2026-07-05T10:25:38.353Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1572gitlab # [ 194.479134] nixbot[3043]: {"ts": "2026-07-05T10:25:38.357Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1573gitlab # [ 194.484503] nixbot[3043]: {"ts": "2026-07-05T10:25:38.362Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1574gitlab # [ 194.492920] nixbot[3043]: {"ts": "2026-07-05T10:25:38.370Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1575gitlab # [ 194.496783] nixbot[3043]: {"ts": "2026-07-05T10:25:38.374Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1576gitlab # [ 194.500875] nixbot[3043]: {"ts": "2026-07-05T10:25:38.376Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1577gitlab # [ 194.505567] nixbot[3043]: {"ts": "2026-07-05T10:25:38.383Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1578gitlab # [ 194.512337] nixbot[3043]: {"ts": "2026-07-05T10:25:38.390Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1579gitlab # [ 194.518385] nixbot[3043]: {"ts": "2026-07-05T10:25:38.396Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1580gitlab # [ 194.524524] nixbot[3043]: {"ts": "2026-07-05T10:25:38.402Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1581gitlab # [ 194.532535] nixbot[3043]: {"ts": "2026-07-05T10:25:38.410Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1582gitlab # [ 194.536315] nixbot[3043]: {"ts": "2026-07-05T10:25:38.414Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1583gitlab # [ 194.540224] nixbot[3043]: {"ts": "2026-07-05T10:25:38.418Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1584gitlab # [ 194.676536] gitaly[1389]: time="2026-07-05T10:25:38.554Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-05T10:25:48.000" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:38.551" grpc.time_ms=2.741 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1585gitlab # [ 194.748761] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:38.626Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 06bf5cb9c56b0e286a64490181153eddf73461ec refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"c5f33f5b34c030f7afa69bce","created_at":"2026-07-05T10:25:36.934Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-2c2ba6371a2f4b7f","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:36.954Z","job_size_bytes":185,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-c5f33f5b34c030f7afa69bce: done: 1.646157 sec","job_status":"done","queue_duration_s":0.025174,"scheduling_latency_s":0.025231,"gitaly_calls":13,"gitaly_duration_s":0.468336,"redis_calls":201,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.20140799999999998,"redis_read_bytes":959,"redis_write_bytes":20011,"redis_cache_calls":14,"redis_cache_duration_s":0.014114,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":34,"redis_feature_flag_duration_s":0.082324,"redis_feature_flag_read_bytes":542,"redis_feature_flag_write_bytes":4562,"redis_queues_calls":8,"redis_queues_duration_s":0.003279,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005905,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.080758,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.015028,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":53,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":53,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.105,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":53,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.533,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.555,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.105,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.444141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001614172,"exclusive_lock_hold_duration_s":0.16655999600000002,"duration_s":1.646157,"completed_at":"2026-07-05T10:25:38.626Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.144495,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1586gitlab # [ 194.832638] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:38.628Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"f3cba3c634ab41af77f0ef5e","created_at":"2026-07-05T10:25:37.350Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-230b965d47ae4a93","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:37.371Z","job_size_bytes":46,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f3cba3c634ab41af77f0ef5e: start","job_status":"start","queue_duration_s":1.25706,"scheduling_latency_s":1.257083}1587gitlab # [ 194.859243] gitaly[1389]: time="2026-07-05T10:25:38.691Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-05T16:25:38.687" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:38.688" grpc.time_ms=2.353 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1588gitlab # [ 194.860793] gitaly[1389]: time="2026-07-05T10:25:38.715Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-05T10:26:08.097" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:38.701" grpc.time_ms=13.555 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1589gitlab # [ 194.880371] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:38.744Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"f3cba3c634ab41af77f0ef5e","created_at":"2026-07-05T10:25:37.350Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-230b965d47ae4a93","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:37.371Z","job_size_bytes":46,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f3cba3c634ab41af77f0ef5e: done: 0.116613 sec","job_status":"done","queue_duration_s":1.25706,"scheduling_latency_s":1.257083,"gitaly_calls":2,"gitaly_duration_s":0.024419,"redis_calls":42,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.017591,"redis_read_bytes":80,"redis_write_bytes":6491,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.007836,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2609,"redis_queues_calls":3,"redis_queues_duration_s":0.001019,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002402,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004751,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001583,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000379657,"duration_s":0.116613,"completed_at":"2026-07-05T10:25:38.744Z","load_balancing_strategy":"primary","db_duration_s":0.019995,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1590gitlab # [ 194.911971] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:38.746Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cbf439f48df2ab84c2e5e3da","created_at":"2026-07-05T10:25:37.385Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-230b965d47ae4a93","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:37.392Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cbf439f48df2ab84c2e5e3da: start","job_status":"start","queue_duration_s":1.353916,"scheduling_latency_s":1.353937}1591gitlab # [ 194.983600] nixbot[3043]: INFO: Started server process [3043]1592gitlab # [ 194.984795] nixbot[3043]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1593gitlab # [ 194.985972] nixbot[3043]: INFO: ::ffff:127.0.0.1:42074 - "GET /health HTTP/1.1" 200 OK1594gitlab # [ 195.011883] systemd[1]: Started nixbot CI service.1595gitlab # [ 195.013126] systemd[1]: Reached target Multi-User System.1596gitlab # [ 195.014278] systemd[1]: Startup finished in 1.698s (kernel) + 4.275s (initrd) + 3min 9.038s (userspace) = 3min 15.011s.1597gitlab # [ 195.022367] gitaly[1389]: time="2026-07-05T10:25:38.900Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 diskcache=f2e8d257-5739-412d-af24-52f57cd8bbde grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-05T16:25:38.773" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:38.774" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1598gitlab # [ 195.038939] gitaly[1389]: time="2026-07-05T10:25:38.916Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1294 command.oublock=0 command.real_time_ms=174 command.spawn_token_fork_ms=87 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-05T16:25:38.773" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:38.774" grpc.time_ms=126.373 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1599gitlab # [ 195.095150] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:38.972Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cbf439f48df2ab84c2e5e3da","created_at":"2026-07-05T10:25:37.385Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-230b965d47ae4a93","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:37.392Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cbf439f48df2ab84c2e5e3da: done: 0.226392 sec","job_status":"done","queue_duration_s":1.353916,"scheduling_latency_s":1.353937,"gitaly_calls":1,"gitaly_duration_s":0.170736,"redis_calls":15,"redis_duration_s":0.004328,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000438,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000615,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001887,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.200740596,"duration_s":0.226392,"completed_at":"2026-07-05T10:25:38.972Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015773,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600gitlab # [ 195.098486] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:38.974Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"d606454c2a1ee324d03dc2bb","created_at":"2026-07-05T10:25:37.595Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-230b965d47ae4a93","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:37.600Z","job_size_bytes":23,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d606454c2a1ee324d03dc2bb: start","job_status":"start","queue_duration_s":1.374464,"scheduling_latency_s":1.374486}1601gitlab # [ 195.206693] gitaly[1389]: time="2026-07-05T10:25:39.084Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZW27F2EH7CG29GEVTC2V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-05T10:26:09.062" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:39.062" grpc.time_ms=21.673 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1602gitlab # [ 195.225308] gitaly[1389]: time="2026-07-05T10:25:39.103Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-05T10:26:09.095" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:39.096" grpc.time_ms=6.921 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603gitlab # [ 195.244543] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:39.121Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"d606454c2a1ee324d03dc2bb","created_at":"2026-07-05T10:25:37.595Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-230b965d47ae4a93","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:37.600Z","job_size_bytes":23,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d606454c2a1ee324d03dc2bb: done: 0.147349 sec","job_status":"done","queue_duration_s":1.374464,"scheduling_latency_s":1.374486,"gitaly_calls":1,"gitaly_duration_s":0.010928,"redis_calls":16,"redis_duration_s":0.005642,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000419,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001553,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002658,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.094,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.094,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.147349,"completed_at":"2026-07-05T10:25:39.121Z","load_balancing_strategy":"primary","db_duration_s":0.094061,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1604gitlab # [ 195.317388] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:39.124Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"71bbf05f04e423fd22053584","created_at":"2026-07-05T10:25:38.519Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-230b965d47ae4a93","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7a4f9e4c29534a6eb37cd197414cd2b8a187b0b89133cf65adee4f4c58d70d8f","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:38.523Z","job_size_bytes":467,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-71bbf05f04e423fd22053584: start","job_status":"start","queue_duration_s":0.601489,"scheduling_latency_s":0.601535}1605gitlab # [ 195.318896] gitaly[1389]: time="2026-07-05T10:25:39.163Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:09.161" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:39.162" grpc.time_ms=0.576 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1606gitlab # [ 195.320145] gitaly[1389]: time="2026-07-05T10:25:39.171Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-05T10:25:49.004" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:39.165" grpc.time_ms=6.55 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1607gitlab # [ 195.406524] gitaly[1389]: time="2026-07-05T10:25:39.284Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-05T10:26:09.282" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:39.283" grpc.time_ms=1.294 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1608gitlab # [ 195.428117] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 4181609gitlab # [ 195.429359] gitaly[1389]: time="2026-07-05T10:25:39.306Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=15 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=15 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-05T10:26:09.288" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:39.288" grpc.time_ms=17.512 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1610gitlab # [ 195.458649] nixbot[3043]: {"ts": "2026-07-05T10:25:39.336Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1611gitlab # [ 195.473625] nixbot[3043]: {"ts": "2026-07-05T10:25:39.351Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1612gitlab # [ 195.525257] gitaly[1389]: time="2026-07-05T10:25:39.402Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:49.232" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:39.392" grpc.time_ms=10.142 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1613gitlab # [ 195.547492] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 571614gitlab # [ 195.549103] nixbot[3043]: {"ts": "2026-07-05T10:25:39.425Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1615gitlab # [ 195.550500] nixbot[3043]: {"ts": "2026-07-05T10:25:39.426Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1616gitlab # [ 195.638685] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 861617gitlab # [ 195.643248] nixbot[3043]: {"ts": "2026-07-05T10:25:39.519Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1618gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds)1619gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1620gitlab # [ 195.701313] nixbot[3043]: INFO: ::ffff:127.0.0.1:42082 - "GET /health HTTP/1.1" 200 OK1621gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds)1622subtest: project discovered and webhook registered1623gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1624gitlab # [ 195.758914] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1071625gitlab # [ 195.769985] nixbot[3043]: {"ts": "2026-07-05T10:25:39.647Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1626gitlab # [ 195.826354] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 441627gitlab # [ 195.835636] nixbot[3043]: {"ts": "2026-07-05T10:25:39.704Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1628gitlab # [ 195.864789] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 1381629gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.18 seconds)1630[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-05T10:25:39.504Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1631gitlab # [ 195.874295] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 431632(finished: subtest: project discovered and webhook registered, in 0.18 seconds)1633subtest: push delivers the webhook1634gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1635gitlab # [ 195.884140] nixbot[3043]: {"ts": "2026-07-05T10:25:39.752Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1636gitlab # [ 195.923879] gitaly[1389]: time="2026-07-05T10:25:39.801Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZWXDMRKZGFH6TSXBXSSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-05T10:26:09.291" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:39.792" grpc.time_ms=8.537 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1637gitlab # [ 195.956146] gitaly[1389]: time="2026-07-05T10:25:39.817Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZWXDMRKZGFH6TSXBXSSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:09.316" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:39.816" grpc.time_ms=0.836 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1638gitlab # [ 195.978154] gitaly[1389]: time="2026-07-05T10:25:39.834Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZWXDMRKZGFH6TSXBXSSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-05T10:25:49.090" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:39.820" grpc.time_ms=14.243 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1639gitlab # [ 196.001353] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1151640gitlab # [ 196.005367] nixbot[3043]: {"ts": "2026-07-05T10:25:39.873Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1641gitlab # [ 196.017637] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 771642gitlab # [ 196.073135] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 731643gitlab # [ 196.077070] nixbot[3043]: {"ts": "2026-07-05T10:25:39.953Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1644gitlab # [ 196.082393] nixbot[3043]: {"ts": "2026-07-05T10:25:39.960Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1645gitlab # [ 196.099252] gitaly[1389]: time="2026-07-05T10:25:39.976Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX1S8N58KSWV3Z2ZCVQ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:39.965" grpc.time_ms=11.485 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1646gitlab # [ 196.101291] gitaly[1389]: time="2026-07-05T10:25:39.979Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=6 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-05T10:26:09.347" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:39.947" grpc.time_ms=31.633 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1647gitlab # [ 196.102992] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.54.0" 811648gitlab # [ 196.122279] gitaly[1389]: time="2026-07-05T10:25:39.999Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZSJCKYMNV44B6SN8YJS4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-05T10:26:09.386" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:39.987" grpc.time_ms=12.554 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1649gitlab # Enumerating objects: 5, done.1650gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1651gitlab # Delta compression using up to 4 threads1652gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1653gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done.1654gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1655gitlab # [ 196.201504] sidekiq-cluster[2102]: {"severity":"WARN","time":"2026-07-05T10:25:40.078Z","class":"CreatePipelineWorker","correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"06bf5cb9c56b0e286a64490181153eddf73461ec","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"06bf5cb9c56b0e286a64490181153eddf73461ec","ref":"refs/heads/master"},"retry":0}1656gitlab # [ 196.213482] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:40.084Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"71bbf05f04e423fd22053584","created_at":"2026-07-05T10:25:38.519Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-230b965d47ae4a93","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7a4f9e4c29534a6eb37cd197414cd2b8a187b0b89133cf65adee4f4c58d70d8f","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:38.523Z","job_size_bytes":467,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-71bbf05f04e423fd22053584: done: 0.960229 sec","job_status":"done","queue_duration_s":0.601489,"scheduling_latency_s":0.601535,"gitaly_calls":7,"gitaly_duration_s":0.101684,"redis_calls":53,"redis_duration_s":0.020185,"redis_read_bytes":171,"redis_write_bytes":4426,"redis_cache_calls":4,"redis_cache_duration_s":0.001307,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.012349,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001409,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002866,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002254,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.679311,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340826,"duration_s":0.960229,"completed_at":"2026-07-05T10:25:40.084Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.132698,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1657gitlab # [ 196.221281] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:40.086Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ec4ce7f3264900a65687d88d","created_at":"2026-07-05T10:25:38.726Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-f0c457b80b3848fa","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:38.732Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ec4ce7f3264900a65687d88d: start","job_status":"start","queue_duration_s":1.354092,"scheduling_latency_s":1.354153}1658gitlab # [ 196.275853] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:40.153Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"ec4ce7f3264900a65687d88d","created_at":"2026-07-05T10:25:38.726Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-f0c457b80b3848fa","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:38.732Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-ec4ce7f3264900a65687d88d: done: 0.067291 sec","job_status":"done","queue_duration_s":1.354092,"scheduling_latency_s":1.354153,"redis_calls":19,"redis_duration_s":0.0067139999999999995,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001028,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000996,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003852,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008833803999999999,"duration_s":0.067291,"completed_at":"2026-07-05T10:25:40.153Z","load_balancing_strategy":"primary","db_duration_s":0.027553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1659gitlab # [ 196.370358] gitaly[1389]: time="2026-07-05T10:25:40.240Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX5AAGMBF3VK881QVEK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=202 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.220" grpc.time_ms=19.828 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1660gitlab # [ 196.375227] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 2301661gitlab # [ 196.376223] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:40.155Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"29445d97d79b3a451fb4d177","created_at":"2026-07-05T10:25:40.136Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-2844558de8a24b1f","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:40.142Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-29445d97d79b3a451fb4d177: start","job_status":"start","queue_duration_s":0.012452,"scheduling_latency_s":0.012473}1662gitlab # [ 196.379711] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:40.257Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"29445d97d79b3a451fb4d177","created_at":"2026-07-05T10:25:40.136Z","trace_propagation_headers":{"sentry-trace":"520e3179d336412a937f65d7bdc78f5f-2844558de8a24b1f","baggage":"sentry-trace_id=520e3179d336412a937f65d7bdc78f5f,sentry-release=Unknown"},"correlation_id":"01KWRWZSJCKYMNV44B6SN8YJS4","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:40.142Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-29445d97d79b3a451fb4d177: done: 0.102513 sec","job_status":"done","queue_duration_s":0.012452,"scheduling_latency_s":0.012473,"redis_calls":11,"redis_duration_s":0.011033,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00325,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007228,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000555,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002052775,"duration_s":0.102513,"completed_at":"2026-07-05T10:25:40.257Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023905,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1663gitlab # [ 196.519536] gitaly[1389]: time="2026-07-05T10:25:40.397Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=48 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-05T10:25:50.006" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:40.346" grpc.time_ms=50.75 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1664gitlab # [ 196.544184] gitaly[1389]: time="2026-07-05T10:25:40.422Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZXCKJ7R71T03NT10P4TQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.353" pid=1389 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1665gitlab # [ 196.545381] gitaly[1389]: time="2026-07-05T10:25:40.422Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZXCKJ7R71T03NT10P4TQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.353" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1666gitlab # [ 196.546273] gitaly[1389]: time="2026-07-05T10:25:40.422Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZXCKJ7R71T03NT10P4TQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.353" grpc.time_ms=69.313 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1667gitlab # [ 196.548241] gitlab-workhorse[1292]: time="2026-07-05T10:25:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491668gitlab # [ 196.549332] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1821669gitlab # [ 196.629147] gitaly[1389]: time="2026-07-05T10:25:40.507Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=332 command.oublock=0 command.real_time_ms=90 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-05T10:26:10.414" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-05T10:25:40.414" grpc.time_ms=92.454 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1670gitlab # [ 196.653519] gitaly[1389]: time="2026-07-05T10:25:40.519Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 duration_ms=230ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/allowed"1671gitlab # [ 196.660570] gitaly[1389]: time="2026-07-05T10:25:40.538Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZXJEX5PSGEAX3376Z2JP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.500" pid=1389 remote_ip="2001:db8:1::1" request_sha=ae99e4e715d0e5164c64595c50e891671ec85921 response_bytes=229 span.kind=server system=grpc1672gitlab # [ 196.679152] gitaly[1389]: time="2026-07-05T10:25:40.538Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1200 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=13 command.system_time_ms=3 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZXJEX5PSGEAX3376Z2JP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.500" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1673gitlab # [ 196.680310] gitlab-workhorse[1292]: time="2026-07-05T10:25:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2291674gitlab # [ 196.681145] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 229 "" "git/2.54.0" 1091675gitlab # [ 196.682109] gitaly[1389]: time="2026-07-05T10:25:40.538Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1200 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=13 command.system_time_ms=3 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZXJEX5PSGEAX3376Z2JP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.500" grpc.time_ms=37.878 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1676gitlab # [ 196.692758] gitaly[1389]: time="2026-07-05T10:25:40.570Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 duration_ms=40ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/pre_receive"1677gitlab # [ 196.694191] gitaly[1389]: time="2026-07-05T10:25:40.571Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:40.285" grpc.time_ms=285.363 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1678gitlab # [ 196.720955] gitaly[1389]: time="2026-07-05T10:25:40.599Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:40.598" grpc.time_ms=0.495 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1679gitlab # [ 196.788566] gitaly[1389]: time="2026-07-05T10:25:40.666Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:40.666" grpc.time_ms=0.335 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1680gitlab # [ 196.797644] gitaly[1389]: time="2026-07-05T10:25:40.675Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZXPYJP2QV43WX66FF4GF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.663" grpc.time_ms=12.389 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1681gitlab # [ 196.801363] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1031682gitlab # [ 196.817938] gitaly[1389]: time="2026-07-05T10:25:40.695Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:40.695" grpc.time_ms=0.367 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1683gitlab # [ 196.885222] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:40.762Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","06bf5cb9c56b0e286a64490181153eddf73461ec a7f9a657ca268f3e3bb99f4a038559b3eee13800 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b5c110d0b65d2cae79802973","created_at":"2026-07-05T10:25:40.756Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-bcc47497692c4260","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:40.760Z","job_size_bytes":185,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b5c110d0b65d2cae79802973: start","job_status":"start","queue_duration_s":0.001475,"scheduling_latency_s":0.001512}1684gitlab # [ 196.908708] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:40.786Z","message":"changes: \"06bf5cb9c56b0e286a64490181153eddf73461ec a7f9a657ca268f3e3bb99f4a038559b3eee13800 refs/heads/master\\n\""}1685gitlab # [ 196.912625] gitaly[1389]: time="2026-07-05T10:25:40.788Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 duration_ms=64ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/post_receive"1686gitlab # [ 196.913763] gitaly[1389]: time="2026-07-05T10:25:40.790Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:40.723" grpc.time_ms=66.985 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1687gitlab # [ 196.930312] gitaly[1389]: time="2026-07-05T10:25:40.807Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 diskcache=4133b011-a580-4f91-880f-138d8f6307b7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.193" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1688gitlab # [ 196.931804] gitaly[1389]: time="2026-07-05T10:25:40.807Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=159 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13461 command.oublock=32 command.real_time_ms=600 command.spawn_token_fork_ms=3 command.system_time_ms=111 command.user_time_ms=48 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.193" grpc.time_ms=613.786 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1689gitlab # [ 196.934523] gitlab-workhorse[1292]: time="2026-07-05T10:25:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521690gitlab # [ 196.934989] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:40 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 7361691gitlab # [ 196.948569] gitaly[1389]: time="2026-07-05T10:25:40.826Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:50.264" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:40.814" grpc.time_ms=11.391 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1692gitlab # [ 196.963744] gitaly[1389]: time="2026-07-05T10:25:40.841Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-05T10:25:50.281" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:40.832" grpc.time_ms=8.808 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1693gitlab # To http://gitlab/root/test-flake.git1694gitlab # 06bf5cb..a7f9a65 master -> master1695gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.17 seconds)1696gitlab: must succeed: git -C /tmp/test-flake rev-parse master1697gitlab # [ 197.049539] gitaly[1389]: time="2026-07-05T10:25:40.927Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-05T10:25:50.008" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:40.919" grpc.time_ms=8.244 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1698gitlab # [ 197.052794] gitaly[1389]: time="2026-07-05T10:25:40.929Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZXVS0JCGPESJAJ1ZY1PJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.920" pid=1389 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1699gitlab # [ 197.053927] gitaly[1389]: time="2026-07-05T10:25:40.930Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZXVS0JCGPESJAJ1ZY1PJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.920" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1700gitlab # [ 197.057249] gitaly[1389]: time="2026-07-05T10:25:40.932Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZXVS0JCGPESJAJ1ZY1PJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:40.920" grpc.time_ms=11.6 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1701gitlab # [ 197.060972] gitlab-workhorse[1292]: time="2026-07-05T10:25:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671702gitlab # [ 197.065287] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 2031703gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.10 seconds)1704gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a7f9a657ca268f3e3bb99f4a038559b3eee13800'1705gitlab # [ 197.156239] gitaly[1389]: time="2026-07-05T10:25:41.034Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-05T10:26:11.023" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:41.024" grpc.time_ms=9.607 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1706gitlab # [ 197.176694] gitaly[1389]: time="2026-07-05T10:25:41.054Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-05T10:25:51.005" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-05T10:25:41.046" grpc.time_ms=8.675 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1707gitlab # [ 197.180109] gitaly[1389]: time="2026-07-05T10:25:41.057Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZY2C1AH9PZ34X59KYPWH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:41.031" pid=1389 remote_ip="2001:db8:1::1" request_sha=9f545399d518d918892145f97b7c8c1af66a499e response_bytes=338 span.kind=server system=grpc1708gitlab # [ 197.181701] gitaly[1389]: time="2026-07-05T10:25:41.057Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1192 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=9 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZY2C1AH9PZ34X59KYPWH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:41.031" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1709gitlab # [ 197.189317] gitlab-workhorse[1292]: time="2026-07-05T10:25:41Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3381710gitlab # [ 197.190400] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:41 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 338 "" "git/2.54.0" 1171711gitlab # [ 197.191320] gitaly[1389]: time="2026-07-05T10:25:41.058Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1192 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=9 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZY2C1AH9PZ34X59KYPWH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:41.031" grpc.time_ms=27.229 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1712gitlab # [ 197.199457] gitaly[1389]: time="2026-07-05T10:25:41.070Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:51.022" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:41.063" grpc.time_ms=6.7 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1713gitlab # [ 197.223127] nixbot[3043]: INFO: ::ffff:127.0.0.1:42096 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a7f9a657ca268f3e3bb99f4a038559b3eee13800 HTTP/1.1" 200 OK1714gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a7f9a657ca268f3e3bb99f4a038559b3eee13800', in 0.15 seconds)1715[]1716gitlab # [ 197.348255] gitaly[1389]: time="2026-07-05T10:25:41.225Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-05T10:25:51.062" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:41.222" grpc.time_ms=2.917 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1717gitlab # [ 197.363557] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:41.240Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","06bf5cb9c56b0e286a64490181153eddf73461ec a7f9a657ca268f3e3bb99f4a038559b3eee13800 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b5c110d0b65d2cae79802973","created_at":"2026-07-05T10:25:40.756Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-bcc47497692c4260","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-05T10:25:40.760Z","job_size_bytes":185,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b5c110d0b65d2cae79802973: done: 0.479062 sec","job_status":"done","queue_duration_s":0.001475,"scheduling_latency_s":0.001512,"gitaly_calls":7,"gitaly_duration_s":0.074439,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.034249,"redis_read_bytes":350,"redis_write_bytes":13068,"redis_cache_calls":9,"redis_cache_duration_s":0.003237,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001398,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6768,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003655,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.015704,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.009811,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.09,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.022,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.022,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.09,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.15951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000609854,"exclusive_lock_hold_duration_s":0.026934708,"duration_s":0.479062,"completed_at":"2026-07-05T10:25:41.240Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.097169,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1718gitlab # [ 197.380430] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:41.242Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"a89563629bb58b054499d6df","created_at":"2026-07-05T10:25:40.843Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-34ff39624d8f4819","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:40.845Z","job_size_bytes":46,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a89563629bb58b054499d6df: start","job_status":"start","queue_duration_s":0.397521,"scheduling_latency_s":0.397546}1719gitlab # [ 197.469186] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:41.260Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"a89563629bb58b054499d6df","created_at":"2026-07-05T10:25:40.843Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-34ff39624d8f4819","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:40.845Z","job_size_bytes":46,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a89563629bb58b054499d6df: done: 0.017967 sec","job_status":"done","queue_duration_s":0.397521,"scheduling_latency_s":0.397546,"redis_calls":9,"redis_duration_s":0.002085,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000226,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000727,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017967,"completed_at":"2026-07-05T10:25:41.260Z","load_balancing_strategy":"primary","db_duration_s":0.001953,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1720gitlab # [ 197.543476] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:41.262Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c205335be5f84a2cd6fce46d","created_at":"2026-07-05T10:25:40.912Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-34ff39624d8f4819","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:40.915Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c205335be5f84a2cd6fce46d: start","job_status":"start","queue_duration_s":0.34705,"scheduling_latency_s":0.347072}1721gitlab # [ 197.544828] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:41.319Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c205335be5f84a2cd6fce46d","created_at":"2026-07-05T10:25:40.912Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-34ff39624d8f4819","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:40.915Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c205335be5f84a2cd6fce46d: done: 0.057467 sec","job_status":"done","queue_duration_s":0.34705,"scheduling_latency_s":0.347072,"gitaly_calls":1,"gitaly_duration_s":0.030114,"redis_calls":12,"redis_duration_s":0.003076,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000674,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001152,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.040093084,"duration_s":0.057467,"completed_at":"2026-07-05T10:25:41.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1722gitlab # [ 197.626143] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:41.321Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b5f6bb0848fe1d77b0b88e87","created_at":"2026-07-05T10:25:41.114Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-34ff39624d8f4819","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:819e35d359ffe59c186af0d24a39a703cda06318ad35445635c4868236395e35","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:41.115Z","job_size_bytes":467,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b5f6bb0848fe1d77b0b88e87: start","job_status":"start","queue_duration_s":0.205366,"scheduling_latency_s":0.205387}1723gitlab # [ 197.627418] gitaly[1389]: time="2026-07-05T10:25:41.306Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 diskcache=697a51a9-ee42-4747-9a67-1d90b5b7f8a7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-05T16:25:41.278" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:41.279" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1724gitlab # [ 197.628320] gitaly[1389]: time="2026-07-05T10:25:41.307Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1647 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-05T16:25:41.278" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:41.279" grpc.time_ms=27.802 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1725gitlab # [ 197.710134] gitaly[1389]: time="2026-07-05T10:25:41.341Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:11.340" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:41.340" grpc.time_ms=0.6 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1726gitlab # [ 197.711786] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:41 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 3441727gitlab # [ 197.713158] gitaly[1389]: time="2026-07-05T10:25:41.353Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-05T10:25:51.182" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:41.343" grpc.time_ms=10.44 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1728gitlab # [ 197.713993] gitaly[1389]: time="2026-07-05T10:25:41.377Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-05T10:26:11.375" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:41.375" grpc.time_ms=1.212 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1729gitlab # [ 197.790502] gitaly[1389]: time="2026-07-05T10:25:41.381Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=59 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-05T10:26:11.379" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:41.379" grpc.time_ms=2.061 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1730gitlab # [ 197.793149] gitaly[1389]: time="2026-07-05T10:25:41.419Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:51.251" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:41.412" grpc.time_ms=7.048 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1731gitlab # [ 197.793806] gitaly[1389]: time="2026-07-05T10:25:41.542Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZYAFVWGWC3CACCWHY1J6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:41.530" grpc.time_ms=11.964 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1732gitlab # [ 197.831198] gitaly[1389]: time="2026-07-05T10:25:41.709Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYNBP0QGX5MNA4H18H7X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:41.694" pid=1389 remote_ip="2001:db8:1::1" request_sha=69dcef142412daaffeeda28f6446e517599d516e response_bytes=214 span.kind=server system=grpc1733gitlab # [ 197.832463] gitaly[1389]: time="2026-07-05T10:25:41.709Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYNBP0QGX5MNA4H18H7X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:41.694" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1734gitlab # [ 197.833351] gitaly[1389]: time="2026-07-05T10:25:41.710Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYNBP0QGX5MNA4H18H7X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:41.694" grpc.time_ms=16.049 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1735gitlab # [ 197.908218] gitlab-workhorse[1292]: time="2026-07-05T10:25:41Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2141736gitlab # [ 197.909271] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:41 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 214 "" "git/2.54.0" 1631737gitlab # [ 198.127156] gitaly[1389]: time="2026-07-05T10:25:42.003Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:11.500" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:42.002" grpc.time_ms=1.328 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1738gitlab # [ 198.229877] gitaly[1389]: time="2026-07-05T10:25:42.107Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-05T10:26:12.089" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:42.090" grpc.time_ms=17.116 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1739gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a7f9a657ca268f3e3bb99f4a038559b3eee13800'1740gitlab # [ 198.292308] gitaly[1389]: time="2026-07-05T10:25:42.169Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:52.152" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:42.163" grpc.time_ms=6.087 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741gitlab # [ 198.320679] gitaly[1389]: time="2026-07-05T10:25:42.198Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=29 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=37 command.spawn_token_fork_ms=4 command.system_time_ms=28 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-05T10:26:12.155" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:42.156" grpc.time_ms=42.05 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1742gitlab # [ 198.336160] gitaly[1389]: time="2026-07-05T10:25:42.211Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZX7BVJAGCZP4JWD39GE2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-05T10:26:12.201" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:42.201" grpc.time_ms=10.022 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1743gitlab # [ 198.379550] sidekiq-cluster[2102]: {"severity":"WARN","time":"2026-07-05T10:25:42.256Z","class":"CreatePipelineWorker","correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"06bf5cb9c56b0e286a64490181153eddf73461ec","after":"a7f9a657ca268f3e3bb99f4a038559b3eee13800","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"a7f9a657ca268f3e3bb99f4a038559b3eee13800","ref":"refs/heads/master"},"retry":0}1744gitlab # [ 198.383778] nixbot[3043]: INFO: ::ffff:127.0.0.1:42110 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a7f9a657ca268f3e3bb99f4a038559b3eee13800 HTTP/1.1" 200 OK1745gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a7f9a657ca268f3e3bb99f4a038559b3eee13800', in 0.10 seconds)1746[]1747gitlab # [ 198.401849] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:42.279Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b5f6bb0848fe1d77b0b88e87","created_at":"2026-07-05T10:25:41.114Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-34ff39624d8f4819","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:819e35d359ffe59c186af0d24a39a703cda06318ad35445635c4868236395e35","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:41.115Z","job_size_bytes":467,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b5f6bb0848fe1d77b0b88e87: done: 0.958164 sec","job_status":"done","queue_duration_s":0.205366,"scheduling_latency_s":0.205387,"gitaly_calls":7,"gitaly_duration_s":0.12061,"redis_calls":33,"redis_duration_s":0.021082,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000754,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.016198,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002606,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001524,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.405,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.402,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.40587,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.958164,"completed_at":"2026-07-05T10:25:42.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.404856,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1748gitlab # [ 198.406958] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:42.281Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"0057d5da-7ef5-41ff-90eb-765df4e26df3\"}"],"class":"WebHookWorker","jid":"3337a91f3ad124d55443392f","created_at":"2026-07-05T10:25:41.204Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-34ff39624d8f4819","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5090259706a3f8fb966d7fb3b504a7ae60277ebf39a76028d77d4372e33d111b","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:41.210Z","job_size_bytes":1749,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3337a91f3ad124d55443392f: start","job_status":"start","queue_duration_s":1.070757,"scheduling_latency_s":1.070793}1749gitlab # [ 198.471990] nixbot[3043]: INFO: ::ffff:127.0.0.1:42124 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1750gitlab # [ 198.485994] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:42.363Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"0057d5da-7ef5-41ff-90eb-765df4e26df3\"}"],"class":"WebHookWorker","jid":"3337a91f3ad124d55443392f","created_at":"2026-07-05T10:25:41.204Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-34ff39624d8f4819","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5090259706a3f8fb966d7fb3b504a7ae60277ebf39a76028d77d4372e33d111b","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:41.210Z","job_size_bytes":1749,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3337a91f3ad124d55443392f: done: 0.08233 sec","job_status":"done","queue_duration_s":1.070757,"scheduling_latency_s":1.070793,"redis_calls":19,"redis_duration_s":0.003803,"redis_read_bytes":31,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000879,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.00031,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.0017,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.007950730999993993,"cpu_s":0.061706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000425194,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.08233,"completed_at":"2026-07-05T10:25:42.363Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1751gitlab # [ 198.567302] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:42.372Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3337a91f3ad124d55443392f"],"class":"WebHooks::LogExecutionWorker","jid":"9e796436e0e6cbe078efb571","created_at":"2026-07-05T10:25:42.352Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-39de88027f194cd3","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:19d741752ddaf3ac7b03cc4d788fc4d54fa1e591e3fa87290c6a591169fade0b","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:42.357Z","job_size_bytes":2507,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9e796436e0e6cbe078efb571: start","job_status":"start","queue_duration_s":0.015101,"scheduling_latency_s":0.015125}1752gitlab # [ 198.598190] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:42.474Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3337a91f3ad124d55443392f"],"class":"WebHooks::LogExecutionWorker","jid":"9e796436e0e6cbe078efb571","created_at":"2026-07-05T10:25:42.352Z","trace_propagation_headers":{"sentry-trace":"1d079f63f2f342d6a90e0b1005c078a5-39de88027f194cd3","baggage":"sentry-trace_id=1d079f63f2f342d6a90e0b1005c078a5,sentry-release=Unknown"},"correlation_id":"01KWRWZX7BVJAGCZP4JWD39GE2","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:19d741752ddaf3ac7b03cc4d788fc4d54fa1e591e3fa87290c6a591169fade0b","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:42.357Z","job_size_bytes":2507,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9e796436e0e6cbe078efb571: done: 0.102141 sec","job_status":"done","queue_duration_s":0.015101,"scheduling_latency_s":0.015125,"redis_calls":10,"redis_duration_s":0.005049,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000654,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000708,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.024,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.024,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001026108,"duration_s":0.102141,"completed_at":"2026-07-05T10:25:42.474Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1753gitlab # [ 198.634974] gitaly[1389]: time="2026-07-05T10:25:42.512Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZZFBCBHWN1DXJSQ058Q0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:42.504" grpc.time_ms=8.505 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1754gitlab # [ 198.640218] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:42 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1351755gitlab # [ 198.693307] gitaly[1389]: time="2026-07-05T10:25:42.571Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZZKM87D2CQQ3ERA2D2WQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:42.562" pid=1389 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1756gitlab # [ 198.694391] gitaly[1389]: time="2026-07-05T10:25:42.571Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZZKM87D2CQQ3ERA2D2WQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:42.562" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1757gitlab # [ 198.695828] gitaly[1389]: time="2026-07-05T10:25:42.573Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZZKM87D2CQQ3ERA2D2WQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:42.562" grpc.time_ms=9.691 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1758gitlab # [ 198.774641] gitlab-workhorse[1292]: time="2026-07-05T10:25:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671759gitlab # [ 198.776106] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 571760gitlab # [ 198.779402] gitaly[1389]: time="2026-07-05T10:25:42.657Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZZPC234RS9Z5WYT2MHP8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:42.647" grpc.time_ms=9.721 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1761gitlab # [ 198.780583] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:42 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 531762gitlab # [ 198.836163] gitaly[1389]: time="2026-07-05T10:25:42.714Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-05T10:25:52.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:42.713" grpc.time_ms=0.247 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763gitlab # [ 198.880859] gitaly[1389]: time="2026-07-05T10:25:42.758Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-05T10:25:52.039" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:42.749" grpc.time_ms=8.741 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1764gitlab # [ 198.908309] gitaly[1389]: time="2026-07-05T10:25:42.786Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZZR548200Y3ZD88ZGTFG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:42.743" pid=1389 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=216 span.kind=server system=grpc1765gitlab # [ 198.910512] gitaly[1389]: time="2026-07-05T10:25:42.787Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=39 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZZR548200Y3ZD88ZGTFG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:42.743" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1766gitlab # [ 198.912657] gitaly[1389]: time="2026-07-05T10:25:42.790Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=39 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZZR548200Y3ZD88ZGTFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-05T10:25:42.743" grpc.time_ms=44.779 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc1767gitlab # [ 198.914773] gitlab-workhorse[1292]: time="2026-07-05T10:25:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161768gitlab # [ 198.915811] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.54.0" 1301769gitlab # [ 198.989422] gitaly[1389]: time="2026-07-05T10:25:42.867Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:42.866" grpc.time_ms=0.483 pid=1389 span.kind=server system=grpc1770gitlab # [ 199.028300] nixbot[3043]: {"ts": "2026-07-05T10:25:42.906Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1771gitlab # [ 199.029941] gitaly[1389]: time="2026-07-05T10:25:42.907Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:42.907" grpc.time_ms=0.364 pid=1389 span.kind=server system=grpc1772gitlab # [ 199.059846] gitaly[1389]: time="2026-07-05T10:25:42.937Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 diskcache=bc2857d9-4f98-46f1-83bc-bc102e4fe500 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-05T10:25:52.051" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:42.761" pid=1389 ref=refs/keep-around/06bf5cb9c56b0e286a64490181153eddf73461ec remote_ip="2001:db8:1::1" revision=06bf5cb9c56b0e286a64490181153eddf73461ec span.kind=server system=grpc username=root1773gitlab # [ 199.061164] gitaly[1389]: time="2026-07-05T10:25:42.938Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=81 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5214 command.oublock=8 command.real_time_ms=164 command.spawn_token_fork_ms=1 command.system_time_ms=51 command.user_time_ms=30 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-05T10:25:52.051" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:42.761" grpc.time_ms=176.032 pid=1389 ref=refs/keep-around/06bf5cb9c56b0e286a64490181153eddf73461ec remote_ip="2001:db8:1::1" revision=06bf5cb9c56b0e286a64490181153eddf73461ec span.kind=server system=grpc username=root1774gitlab # [ 199.064393] gitaly[1389]: time="2026-07-05T10:25:42.942Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:12.441" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:42.941" grpc.time_ms=0.507 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775gitlab # [ 199.075145] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:42.952Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"592137c96386dcc086ff3e71","created_at":"2026-07-05T10:25:42.945Z","trace_propagation_headers":{"sentry-trace":"20739ffb06db4defbff4f51e256e50fe-5448c02d804c4dfc","baggage":"sentry-trace_id=20739ffb06db4defbff4f51e256e50fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZYYN1EA3X615Z4S15545","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:42.951Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-592137c96386dcc086ff3e71: start","job_status":"start","queue_duration_s":0.001469,"scheduling_latency_s":0.001505}1776gitlab # [ 199.130122] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:43.007Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"592137c96386dcc086ff3e71","created_at":"2026-07-05T10:25:42.945Z","trace_propagation_headers":{"sentry-trace":"20739ffb06db4defbff4f51e256e50fe-5448c02d804c4dfc","baggage":"sentry-trace_id=20739ffb06db4defbff4f51e256e50fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZYYN1EA3X615Z4S15545","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:42.951Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-592137c96386dcc086ff3e71: done: 0.054877 sec","job_status":"done","queue_duration_s":0.001469,"scheduling_latency_s":0.001505,"redis_calls":19,"redis_duration_s":0.005272,"redis_read_bytes":145,"redis_write_bytes":2300,"redis_feature_flag_calls":11,"redis_feature_flag_duration_s":0.002936,"redis_feature_flag_read_bytes":138,"redis_feature_flag_write_bytes":1482,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001599,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000737,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000532191,"duration_s":0.054877,"completed_at":"2026-07-05T10:25:43.007Z","load_balancing_strategy":"primary","db_duration_s":0.010281,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1777gitlab # [ 199.151544] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:43.012Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9cfbc600a49fdfb0a4bd0b4c","created_at":"2026-07-05T10:25:42.953Z","trace_propagation_headers":{"sentry-trace":"20739ffb06db4defbff4f51e256e50fe-5448c02d804c4dfc","baggage":"sentry-trace_id=20739ffb06db4defbff4f51e256e50fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZYYN1EA3X615Z4S15545","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:42.958Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9cfbc600a49fdfb0a4bd0b4c: start","job_status":"start","queue_duration_s":0.053418,"scheduling_latency_s":0.053447}1778gitlab # [ 199.236155] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/06bf5cb9c56b0e286a64490181153eddf73461ec HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 12381779gitlab # [ 199.239224] nixbot[3043]: {"ts": "2026-07-05T10:25:43.086Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/06bf5cb9c56b0e286a64490181153eddf73461ec \"HTTP/1.1 201 Created\""}1780gitlab # [ 199.240173] nixbot[3043]: {"ts": "2026-07-05T10:25:43.093Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-06bf5cb9c56b-2a8f852f", "argv0": "bwrap"}1781gitlab # [ 199.240770] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:43.072Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9cfbc600a49fdfb0a4bd0b4c","created_at":"2026-07-05T10:25:42.953Z","trace_propagation_headers":{"sentry-trace":"20739ffb06db4defbff4f51e256e50fe-5448c02d804c4dfc","baggage":"sentry-trace_id=20739ffb06db4defbff4f51e256e50fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZYYN1EA3X615Z4S15545","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:42.958Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9cfbc600a49fdfb0a4bd0b4c: done: 0.05994 sec","job_status":"done","queue_duration_s":0.053418,"scheduling_latency_s":0.053447,"redis_calls":15,"redis_duration_s":0.003751,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000705,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001526,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000592533,"duration_s":0.05994,"completed_at":"2026-07-05T10:25:43.072Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1782gitlab # [ 199.328268] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:43.074Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"30505460e3a332bcab9dc419","created_at":"2026-07-05T10:25:43.063Z","trace_propagation_headers":{"sentry-trace":"20739ffb06db4defbff4f51e256e50fe-5448c02d804c4dfc","baggage":"sentry-trace_id=20739ffb06db4defbff4f51e256e50fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZYYN1EA3X615Z4S15545","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.069Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-30505460e3a332bcab9dc419: start","job_status":"start","queue_duration_s":0.004573,"scheduling_latency_s":0.004594}1783gitlab # [ 199.340987] gitaly[1389]: time="2026-07-05T10:25:43.218Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-05T10:26:13.216" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:43.217" grpc.time_ms=0.795 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1784gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a7f9a657ca268f3e3bb99f4a038559b3eee13800'1785gitlab # [ 199.369564] gitaly[1389]: time="2026-07-05T10:25:43.247Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:13.246" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:43.246" grpc.time_ms=0.59 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1786gitlab # [ 199.390317] gitaly[1389]: time="2026-07-05T10:25:43.268Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRWZYYN1EA3X615Z4S15545 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:13.266" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:43.267" grpc.time_ms=0.749 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1787gitlab # [ 199.402154] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:43.279Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"30505460e3a332bcab9dc419","created_at":"2026-07-05T10:25:43.063Z","trace_propagation_headers":{"sentry-trace":"20739ffb06db4defbff4f51e256e50fe-5448c02d804c4dfc","baggage":"sentry-trace_id=20739ffb06db4defbff4f51e256e50fe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZYYN1EA3X615Z4S15545","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.069Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-30505460e3a332bcab9dc419: done: 0.205505 sec","job_status":"done","queue_duration_s":0.004573,"scheduling_latency_s":0.004594,"gitaly_calls":3,"gitaly_duration_s":0.011579,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.01007,"redis_read_bytes":208,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001186,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001714,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001716,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000333,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001908,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00068,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000222,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.143055,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.179482589,"duration_s":0.205505,"completed_at":"2026-07-05T10:25:43.279Z","load_balancing_strategy":"primary","db_duration_s":0.029356,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1788gitlab # [ 199.484406] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:43.285Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1304f312641defd9220fdfa6","created_at":"2026-07-05T10:25:43.197Z","trace_propagation_headers":{"sentry-trace":"20739ffb06db4defbff4f51e256e50fe-5ad103f457da4e04","baggage":"sentry-trace_id=20739ffb06db4defbff4f51e256e50fe,sentry-release=Unknown"},"correlation_id":"01KWRWZYYN1EA3X615Z4S15545","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.202Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1304f312641defd9220fdfa6: start","job_status":"start","queue_duration_s":0.083629,"scheduling_latency_s":0.083651}1789gitlab # [ 199.485844] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:43.317Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1304f312641defd9220fdfa6","created_at":"2026-07-05T10:25:43.197Z","trace_propagation_headers":{"sentry-trace":"20739ffb06db4defbff4f51e256e50fe-5ad103f457da4e04","baggage":"sentry-trace_id=20739ffb06db4defbff4f51e256e50fe,sentry-release=Unknown"},"correlation_id":"01KWRWZYYN1EA3X615Z4S15545","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.202Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1304f312641defd9220fdfa6: done: 0.031602 sec","job_status":"done","queue_duration_s":0.083629,"scheduling_latency_s":0.083651,"redis_calls":10,"redis_duration_s":0.00232,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000544,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033887,"duration_s":0.031602,"completed_at":"2026-07-05T10:25:43.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003235,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1790gitlab # [ 199.619162] nixbot[3043]: INFO: ::ffff:127.0.0.1:42134 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a7f9a657ca268f3e3bb99f4a038559b3eee13800 HTTP/1.1" 200 OK1791gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a7f9a657ca268f3e3bb99f4a038559b3eee13800', in 0.23 seconds)1792[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-05T10:25:42.896857Z', 'started_at': None, 'finished_at': None}]1793(finished: subtest: push delivers the webhook, in 3.74 seconds)1794subtest: eval and build post commit statuses1795gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses1796gitlab # [ 199.656244] gitaly[1389]: time="2026-07-05T10:25:43.532Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:13.030" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:43.531" grpc.time_ms=0.774 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1797gitlab # [ 199.680762] gitaly[1389]: time="2026-07-05T10:25:43.558Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:13.057" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:43.557" grpc.time_ms=0.82 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798gitlab # [ 199.694813] gitaly[1389]: time="2026-07-05T10:25:43.572Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-05T10:26:13.062" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:43.563" grpc.time_ms=9.595 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799gitlab # [ 199.732788] gitaly[1389]: time="2026-07-05T10:25:43.610Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:53.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:43.602" grpc.time_ms=8.075 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800gitlab # [ 199.767133] gitaly[1389]: time="2026-07-05T10:25:43.645Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:53.268" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:43.638" grpc.time_ms=6.295 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab # [ 199.792184] gitaly[1389]: time="2026-07-05T10:25:43.669Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX00MD40WAENSA6HQAZ2PR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:13.168" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:43.668" grpc.time_ms=0.765 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1802gitlab # [ 199.801060] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:43.678Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"adf297fb46035028549a8e45","created_at":"2026-07-05T10:25:43.672Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-5ae62d066ecc49db","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.675Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-adf297fb46035028549a8e45: start","job_status":"start","queue_duration_s":0.002887,"scheduling_latency_s":0.002924}1803gitlab # [ 199.828665] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/06bf5cb9c56b0e286a64490181153eddf73461ec HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 1951804gitlab # [ 199.830167] gitaly[1389]: time="2026-07-05T10:25:43.706Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-05T10:25:53.104" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:43.705" grpc.time_ms=1.223 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805gitlab # [ 199.830995] nixbot[3043]: {"ts": "2026-07-05T10:25:43.708Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/06bf5cb9c56b0e286a64490181153eddf73461ec \"HTTP/1.1 201 Created\""}1806gitlab # [ 199.855252] gitaly[1389]: time="2026-07-05T10:25:43.733Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-05T10:25:53.116" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:43.716" grpc.time_ms=11.016 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807gitlab # [ 199.863829] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 1761808gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses, in 0.27 seconds)1809[]1810gitlab # [ 199.921244] gitaly[1389]: time="2026-07-05T10:25:43.799Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:43.798" grpc.time_ms=0.392 pid=1389 span.kind=server system=grpc1811gitlab # [ 199.941794] gitaly[1389]: time="2026-07-05T10:25:43.819Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:43.816" grpc.time_ms=2.113 pid=1389 span.kind=server system=grpc1812gitlab # [ 199.959303] gitaly[1389]: time="2026-07-05T10:25:43.837Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX00TXP2R5HWZK9PS95P82 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:13.334" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:43.834" grpc.time_ms=1.918 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab # [ 199.984370] gitaly[1389]: time="2026-07-05T10:25:43.862Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:43.861" grpc.time_ms=0.515 pid=1389 span.kind=server system=grpc1814gitlab # [ 199.996180] gitaly[1389]: time="2026-07-05T10:25:43.874Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:43.873" grpc.time_ms=0.403 pid=1389 span.kind=server system=grpc1815gitlab # [ 200.000697] gitaly[1389]: time="2026-07-05T10:25:43.878Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRX00TXP2R5HWZK9PS95P82 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:53.153" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:43.864" grpc.time_ms=14.521 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1816gitlab # [ 200.046167] gitaly[1389]: time="2026-07-05T10:25:43.923Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H diskcache=8aa5b683-c31e-403f-adda-3197b6c95456 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-05T10:25:53.137" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:43.739" pid=1389 ref=refs/keep-around/a7f9a657ca268f3e3bb99f4a038559b3eee13800 remote_ip="2001:db8:1::1" revision=a7f9a657ca268f3e3bb99f4a038559b3eee13800 span.kind=server system=grpc username=root1817gitlab # [ 200.047424] gitaly[1389]: time="2026-07-05T10:25:43.924Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=119 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5199 command.oublock=8 command.real_time_ms=176 command.spawn_token_fork_ms=30 command.system_time_ms=104 command.user_time_ms=15 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-05T10:25:53.137" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-05T10:25:43.739" grpc.time_ms=184.422 pid=1389 ref=refs/keep-around/a7f9a657ca268f3e3bb99f4a038559b3eee13800 remote_ip="2001:db8:1::1" revision=a7f9a657ca268f3e3bb99f4a038559b3eee13800 span.kind=server system=grpc username=root1818gitlab # [ 200.053814] gitaly[1389]: time="2026-07-05T10:25:43.931Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:13.430" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:43.931" grpc.time_ms=0.492 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1819gitlab # [ 200.055812] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/06bf5cb9c56b0e286a64490181153eddf73461ec HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1591820gitlab # [ 200.057955] nixbot[3043]: {"ts": "2026-07-05T10:25:43.934Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/06bf5cb9c56b0e286a64490181153eddf73461ec \"HTTP/1.1 201 Created\""}1821gitlab # [ 200.127986] gitaly[1389]: time="2026-07-05T10:25:44.006Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:44.005" grpc.time_ms=0.458 pid=1389 span.kind=server system=grpc1822gitlab # [ 200.153817] gitaly[1389]: time="2026-07-05T10:25:44.031Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP diskcache=b4c00031-09b6-4459-b324-5625c8048e89 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:13.769" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:43.770" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1823gitlab # [ 200.170246] gitaly[1389]: time="2026-07-05T10:25:44.048Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=119 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6959 command.oublock=0 command.real_time_ms=251 command.spawn_token_fork_ms=0 command.system_time_ms=94 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:13.769" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:43.770" grpc.time_ms=261.077 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1824gitlab # [ 200.187752] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:44 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a7f9a657ca268f3e3bb99f4a038559b3eee13800 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 11501825gitlab # [ 200.189380] nixbot[3043]: {"ts": "2026-07-05T10:25:44.066Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a7f9a657ca268f3e3bb99f4a038559b3eee13800 \"HTTP/1.1 201 Created\""}1826gitlab # [ 200.195624] nixbot[3043]: {"ts": "2026-07-05T10:25:44.073Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-a7f9a657ca26-533bb012", "argv0": "bwrap"}1827gitlab # [ 200.276291] gitaly[1389]: time="2026-07-05T10:25:44.153Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-05T10:26:14.152" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:44.152" grpc.time_ms=0.88 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1828gitlab # [ 200.299337] systemd[1]: Started Nix Daemon.1829gitlab # [ 200.324506] gitaly[1389]: time="2026-07-05T10:25:44.202Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:14.201" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:44.201" grpc.time_ms=0.777 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1830gitlab # [ 200.335792] nix-daemon[3472]: accepted connection from pid 3470, user nixbot1831gitlab # [ 200.340123] nix-daemon[3472]: accepted connection from pid 3471, user nixbot1832gitlab # [ 200.380760] gitaly[1389]: time="2026-07-05T10:25:44.258Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:14.256" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:44.257" grpc.time_ms=0.78 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1833gitlab # [ 200.386451] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.264Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"1b4dcf7b946e14251c754bdb","created_at":"2026-07-05T10:25:44.262Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"adf297fb46035028549a8e45","job_size_bytes":3,"pid":2102,"job_status":"deduplicated","message":"PipelineProcessWorker JID-1b4dcf7b946e14251c754bdb: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1834gitlab # [ 200.388136] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.266Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"adf297fb46035028549a8e45","created_at":"2026-07-05T10:25:43.672Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-5ae62d066ecc49db","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.675Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2102,"job_status":"rescheduled","message":"PipelineProcessWorker JID-adf297fb46035028549a8e45: rescheduled"}1835gitlab # [ 200.394107] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.271Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"adf297fb46035028549a8e45","created_at":"2026-07-05T10:25:43.672Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-5ae62d066ecc49db","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.675Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-adf297fb46035028549a8e45: done: 0.593274 sec","job_status":"done","queue_duration_s":0.002887,"scheduling_latency_s":0.002924,"gitaly_calls":4,"gitaly_duration_s":0.368905,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.025759999999999998,"redis_read_bytes":267,"redis_write_bytes":24007,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00278,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001768,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.005471,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.003564,"redis_queues_read_bytes":27,"redis_queues_write_bytes":12386,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006071,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000246,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000621,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002459,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.016,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.170957,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.56511009,"duration_s":0.593274,"completed_at":"2026-07-05T10:25:44.271Z","load_balancing_strategy":"primary","db_duration_s":0.049787,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1836gitlab # [ 200.468621] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.273Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9626569a94c863311486b9e8","created_at":"2026-07-05T10:25:43.757Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.761Z","job_size_bytes":23,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9626569a94c863311486b9e8: start","job_status":"start","queue_duration_s":0.512018,"scheduling_latency_s":0.512041}1837gitlab # [ 200.494071] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.371Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9626569a94c863311486b9e8","created_at":"2026-07-05T10:25:43.757Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.761Z","job_size_bytes":23,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9626569a94c863311486b9e8: done: 0.097686 sec","job_status":"done","queue_duration_s":0.512018,"scheduling_latency_s":0.512041,"redis_calls":15,"redis_duration_s":0.004734,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.00058,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002553,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000528,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000657,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071522,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000810997,"duration_s":0.097686,"completed_at":"2026-07-05T10:25:44.371Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014747,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1838gitlab # [ 200.581460] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.374Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d4880f3a9a378f8d717b39ee","created_at":"2026-07-05T10:25:44.350Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-7c81c3dd70af4812","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:7d59d62d4d4de4723319f82028c16736273537b327681841f0c809d5302b4bc8","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.356Z","job_size_bytes":464,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d4880f3a9a378f8d717b39ee: start","job_status":"start","queue_duration_s":0.018164,"scheduling_latency_s":0.018215}1839gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses1840gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses, in 0.10 seconds)1841[{'id': 3, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-05T10:25:44.040Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1842gitlab # [ 200.593421] sidekiq-cluster[2102]: {"severity":"WARN","time":"2026-07-05T10:25:44.421Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d4880f3a9a378f8d717b39ee","created_at":"2026-07-05T10:25:44.350Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-7c81c3dd70af4812","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:7d59d62d4d4de4723319f82028c16736273537b327681841f0c809d5302b4bc8","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.356Z","job_size_bytes":464,"pid":2102,"sidekiq_tid":"m2ye","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d4880f3a9a378f8d717b39ee: fail: 0.046013 sec","job_status":"fail","queue_duration_s":0.018164,"scheduling_latency_s":0.018215,"redis_calls":7,"redis_duration_s":0.001927,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001328,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000599,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046013,"completed_at":"2026-07-05T10:25:44.420Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.005188,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1843gitlab # [ 201.265770] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.424Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cf4796f9c75fc20808cc9330","created_at":"2026-07-05T10:25:43.942Z","trace_propagation_headers":{"sentry-trace":"b24493c3ae6d429582466de807581936-a12b5c4ab31b457d","baggage":"sentry-trace_id=b24493c3ae6d429582466de807581936,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX0002NS1TWZX5VMRHF39H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.946Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cf4796f9c75fc20808cc9330: start","job_status":"start","queue_duration_s":0.477665,"scheduling_latency_s":0.477718}1844gitlab # [ 201.266805] gitaly[1389]: time="2026-07-05T10:25:44.667Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-05T10:26:14.665" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:44.665" grpc.time_ms=2.12 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845gitlab # [ 201.268379] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 691846gitlab # [ 201.269391] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.565Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cf4796f9c75fc20808cc9330","created_at":"2026-07-05T10:25:43.942Z","trace_propagation_headers":{"sentry-trace":"b24493c3ae6d429582466de807581936-a12b5c4ab31b457d","baggage":"sentry-trace_id=b24493c3ae6d429582466de807581936,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX0002NS1TWZX5VMRHF39H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.946Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cf4796f9c75fc20808cc9330: done: 0.141051 sec","job_status":"done","queue_duration_s":0.477665,"scheduling_latency_s":0.477718,"redis_calls":7,"redis_duration_s":0.001864,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001392,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.116,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.115,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.141051,"completed_at":"2026-07-05T10:25:44.564Z","load_balancing_strategy":"primary","db_duration_s":0.11589,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1847gitlab # [ 201.353467] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.566Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"432029de3b2aede215c39780","created_at":"2026-07-05T10:25:43.947Z","trace_propagation_headers":{"sentry-trace":"b24493c3ae6d429582466de807581936-a12b5c4ab31b457d","baggage":"sentry-trace_id=b24493c3ae6d429582466de807581936,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX0002NS1TWZX5VMRHF39H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.949Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-432029de3b2aede215c39780: start","job_status":"start","queue_duration_s":0.617312,"scheduling_latency_s":0.617337}1848gitlab # [ 201.355234] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.609Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"432029de3b2aede215c39780","created_at":"2026-07-05T10:25:43.947Z","trace_propagation_headers":{"sentry-trace":"b24493c3ae6d429582466de807581936-a12b5c4ab31b457d","baggage":"sentry-trace_id=b24493c3ae6d429582466de807581936,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX0002NS1TWZX5VMRHF39H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:43.949Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-432029de3b2aede215c39780: done: 0.043211 sec","job_status":"done","queue_duration_s":0.617312,"scheduling_latency_s":0.617337,"redis_calls":10,"redis_duration_s":0.004531,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003027,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001504,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043211,"completed_at":"2026-07-05T10:25:44.609Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009518,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1849gitlab # [ 201.426358] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.611Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"74ceda49f1cdd3ea0f440e26","created_at":"2026-07-05T10:25:44.046Z","trace_propagation_headers":{"sentry-trace":"b24493c3ae6d429582466de807581936-a12b5c4ab31b457d","baggage":"sentry-trace_id=b24493c3ae6d429582466de807581936,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX0002NS1TWZX5VMRHF39H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.050Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-74ceda49f1cdd3ea0f440e26: start","job_status":"start","queue_duration_s":0.560621,"scheduling_latency_s":0.560648}1850gitlab # [ 201.495590] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.728Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"74ceda49f1cdd3ea0f440e26","created_at":"2026-07-05T10:25:44.046Z","trace_propagation_headers":{"sentry-trace":"b24493c3ae6d429582466de807581936-a12b5c4ab31b457d","baggage":"sentry-trace_id=b24493c3ae6d429582466de807581936,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX0002NS1TWZX5VMRHF39H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.050Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-74ceda49f1cdd3ea0f440e26: done: 0.116711 sec","job_status":"done","queue_duration_s":0.560621,"scheduling_latency_s":0.560648,"gitaly_calls":3,"gitaly_duration_s":0.010388,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007586,"redis_read_bytes":99,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000961,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002017,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000278,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001583,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000272,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000502,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076443,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09603958,"duration_s":0.116711,"completed_at":"2026-07-05T10:25:44.728Z","load_balancing_strategy":"primary","db_duration_s":0.029657,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1851gitlab # [ 201.570286] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.729Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"519b11707d01785a93636a1b","created_at":"2026-07-05T10:25:44.087Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.093Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-519b11707d01785a93636a1b: start","job_status":"start","queue_duration_s":0.636332,"scheduling_latency_s":0.636354}1852gitlab # [ 201.571720] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.752Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"519b11707d01785a93636a1b","created_at":"2026-07-05T10:25:44.087Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.093Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-519b11707d01785a93636a1b: done: 0.022935 sec","job_status":"done","queue_duration_s":0.636332,"scheduling_latency_s":0.636354,"redis_calls":10,"redis_duration_s":0.002226,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000391949,"duration_s":0.022935,"completed_at":"2026-07-05T10:25:44.752Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002858,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1853gitlab # [ 201.677444] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.754Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"01199f0d091836c7755f7bbf","created_at":"2026-07-05T10:25:44.108Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.114Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-01199f0d091836c7755f7bbf: start","job_status":"start","queue_duration_s":0.639237,"scheduling_latency_s":0.639257}1854gitlab # [ 201.678945] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.784Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"01199f0d091836c7755f7bbf","created_at":"2026-07-05T10:25:44.108Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.114Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-01199f0d091836c7755f7bbf: done: 0.03024 sec","job_status":"done","queue_duration_s":0.639237,"scheduling_latency_s":0.639257,"redis_calls":10,"redis_duration_s":0.002301,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000395581,"duration_s":0.03024,"completed_at":"2026-07-05T10:25:44.784Z","load_balancing_strategy":"primary","db_duration_s":0.005921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1855gitlab # [ 201.759757] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.786Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2a081719868bd54cfaf16313","created_at":"2026-07-05T10:25:44.125Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.129Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2a081719868bd54cfaf16313: start","job_status":"start","queue_duration_s":0.656406,"scheduling_latency_s":0.656426}1856gitlab # [ 201.761348] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.813Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2a081719868bd54cfaf16313","created_at":"2026-07-05T10:25:44.125Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.129Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2a081719868bd54cfaf16313: done: 0.027412 sec","job_status":"done","queue_duration_s":0.656406,"scheduling_latency_s":0.656426,"redis_calls":13,"redis_duration_s":0.003333,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000492,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001715,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000688,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000402006,"duration_s":0.027412,"completed_at":"2026-07-05T10:25:44.813Z","load_balancing_strategy":"primary","db_duration_s":0.003958,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1857gitlab # [ 201.846661] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.814Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7aa77fc85c500761b56000c5","created_at":"2026-07-05T10:25:44.130Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.133Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7aa77fc85c500761b56000c5: start","job_status":"start","queue_duration_s":0.681079,"scheduling_latency_s":0.6811}1858gitlab # [ 201.848311] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.837Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7aa77fc85c500761b56000c5","created_at":"2026-07-05T10:25:44.130Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.133Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7aa77fc85c500761b56000c5: done: 0.022943 sec","job_status":"done","queue_duration_s":0.681079,"scheduling_latency_s":0.6811,"redis_calls":10,"redis_duration_s":0.002298,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000484,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001366,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000334121,"duration_s":0.022943,"completed_at":"2026-07-05T10:25:44.837Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1859gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses1860gitlab # [ 201.955357] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.839Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d3e9d30b3c0f82541efeaca2","created_at":"2026-07-05T10:25:44.134Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.138Z","job_size_bytes":19,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d3e9d30b3c0f82541efeaca2: start","job_status":"start","queue_duration_s":0.700291,"scheduling_latency_s":0.700364}1861gitlab # [ 201.956939] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.868Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d3e9d30b3c0f82541efeaca2","created_at":"2026-07-05T10:25:44.134Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.138Z","job_size_bytes":19,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d3e9d30b3c0f82541efeaca2: done: 0.028878 sec","job_status":"done","queue_duration_s":0.700291,"scheduling_latency_s":0.700364,"redis_calls":11,"redis_duration_s":0.005173,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001215,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003067,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033887,"duration_s":0.028878,"completed_at":"2026-07-05T10:25:44.868Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005607,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1862gitlab # [ 202.031122] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.869Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0bb116d91ceac70c75472f73","created_at":"2026-07-05T10:25:44.141Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.145Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0bb116d91ceac70c75472f73: start","job_status":"start","queue_duration_s":0.724397,"scheduling_latency_s":0.724418}1863gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses, in 0.12 seconds)1864[{'id': 3, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-05T10:25:44.040Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1865gitlab # [ 202.042045] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.898Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0bb116d91ceac70c75472f73","created_at":"2026-07-05T10:25:44.141Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.145Z","job_size_bytes":65,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0bb116d91ceac70c75472f73: done: 0.029519 sec","job_status":"done","queue_duration_s":0.724397,"scheduling_latency_s":0.724418,"redis_calls":10,"redis_duration_s":0.002721,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000503,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001648,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00057,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384406,"duration_s":0.029519,"completed_at":"2026-07-05T10:25:44.898Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1866gitlab # [ 202.117880] gitaly[1389]: time="2026-07-05T10:25:44.696Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:14.694" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:44.695" grpc.time_ms=0.728 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 202.201309] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 721868gitlab # [ 202.202858] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.900Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f36e7797b822c96fc1fc647a","created_at":"2026-07-05T10:25:44.145Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.147Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f36e7797b822c96fc1fc647a: start","job_status":"start","queue_duration_s":0.7535,"scheduling_latency_s":0.753523}1869gitlab # [ 202.203973] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.927Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f36e7797b822c96fc1fc647a","created_at":"2026-07-05T10:25:44.145Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.147Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f36e7797b822c96fc1fc647a: done: 0.026953 sec","job_status":"done","queue_duration_s":0.7535,"scheduling_latency_s":0.753523,"redis_calls":7,"redis_duration_s":0.001728,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001269,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026953,"completed_at":"2026-07-05T10:25:44.927Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002985,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1870gitlab # [ 202.231345] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.929Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cbeab8182d9ce011b23f7bc9","created_at":"2026-07-05T10:25:44.227Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.232Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cbeab8182d9ce011b23f7bc9: start","job_status":"start","queue_duration_s":0.697015,"scheduling_latency_s":0.697039}1871gitlab # [ 202.301333] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.957Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cbeab8182d9ce011b23f7bc9","created_at":"2026-07-05T10:25:44.227Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.232Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cbeab8182d9ce011b23f7bc9: done: 0.028819 sec","job_status":"done","queue_duration_s":0.697015,"scheduling_latency_s":0.697039,"redis_calls":10,"redis_duration_s":0.0024,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000485,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001465,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00045,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000402845,"duration_s":0.028819,"completed_at":"2026-07-05T10:25:44.957Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1872gitlab # [ 202.377086] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.959Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"90bd6c822337bdffb4bd410c","created_at":"2026-07-05T10:25:44.232Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.236Z","job_size_bytes":6,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-90bd6c822337bdffb4bd410c: start","job_status":"start","queue_duration_s":0.72292,"scheduling_latency_s":0.722942}1873gitlab # [ 202.378399] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.979Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"90bd6c822337bdffb4bd410c","created_at":"2026-07-05T10:25:44.232Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.236Z","job_size_bytes":6,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-90bd6c822337bdffb4bd410c: done: 0.019821 sec","job_status":"done","queue_duration_s":0.72292,"scheduling_latency_s":0.722942,"redis_calls":10,"redis_duration_s":0.00233,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000499,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000502,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350324,"duration_s":0.019821,"completed_at":"2026-07-05T10:25:44.979Z","load_balancing_strategy":"primary","db_duration_s":0.001596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1874gitlab # [ 202.453960] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:44.980Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7cbeb830608d252d5691e993","created_at":"2026-07-05T10:25:44.266Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.267Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7cbeb830608d252d5691e993: start","job_status":"start","queue_duration_s":0.712796,"scheduling_latency_s":0.712817}1875gitlab # [ 202.454737] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:45.051Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7cbeb830608d252d5691e993","created_at":"2026-07-05T10:25:44.266Z","trace_propagation_headers":{"sentry-trace":"2831e81ebf6e4308ae6b4a67e53ee26a-a06446195be6411f","baggage":"sentry-trace_id=2831e81ebf6e4308ae6b4a67e53ee26a,sentry-release=Unknown"},"correlation_id":"01KWRX00JP53ZAV63DY0X65ZYP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.267Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7cbeb830608d252d5691e993: done: 0.070519 sec","job_status":"done","queue_duration_s":0.712796,"scheduling_latency_s":0.712817,"gitaly_calls":2,"gitaly_duration_s":0.005907,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003365,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000639,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001455,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000243,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.048999803,"duration_s":0.070519,"completed_at":"2026-07-05T10:25:45.050Z","load_balancing_strategy":"primary","db_duration_s":0.013089,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1876gitlab # [ 202.541379] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:45.052Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"c07267eda1a034948e42da97","created_at":"2026-07-05T10:25:44.655Z","trace_propagation_headers":{"sentry-trace":"b24493c3ae6d429582466de807581936-dd67e1bd28cf4b4a","baggage":"sentry-trace_id=b24493c3ae6d429582466de807581936,sentry-release=Unknown"},"correlation_id":"01KWRX0002NS1TWZX5VMRHF39H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.657Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c07267eda1a034948e42da97: start","job_status":"start","queue_duration_s":0.39519,"scheduling_latency_s":0.395215}1877gitlab # [ 202.545730] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:45.079Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"c07267eda1a034948e42da97","created_at":"2026-07-05T10:25:44.655Z","trace_propagation_headers":{"sentry-trace":"b24493c3ae6d429582466de807581936-dd67e1bd28cf4b4a","baggage":"sentry-trace_id=b24493c3ae6d429582466de807581936,sentry-release=Unknown"},"correlation_id":"01KWRX0002NS1TWZX5VMRHF39H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:44.657Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c07267eda1a034948e42da97: done: 0.026471 sec","job_status":"done","queue_duration_s":0.39519,"scheduling_latency_s":0.395215,"redis_calls":7,"redis_duration_s":0.00179,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001294,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026471,"completed_at":"2026-07-05T10:25:45.079Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1878gitlab # [ 202.627584] gitaly[1389]: time="2026-07-05T10:25:44.719Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0002NS1TWZX5VMRHF39H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:14.718" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:44.719" grpc.time_ms=0.697 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879gitlab # [ 202.699518] gitaly[1389]: time="2026-07-05T10:25:44.843Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX01V1KAAY7MZBE9RPDD3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:14.341" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:44.841" grpc.time_ms=1.316 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab # [ 202.701098] gitaly[1389]: time="2026-07-05T10:25:45.015Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-05T10:26:15.013" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:45.014" grpc.time_ms=0.98 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1881gitlab # [ 202.701933] gitaly[1389]: time="2026-07-05T10:25:45.041Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX00JP53ZAV63DY0X65ZYP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:15.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:45.040" grpc.time_ms=0.581 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1882gitlab # [ 202.703154] gitaly[1389]: time="2026-07-05T10:25:45.973Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX02Y00CAVQFMCXABABZZ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:15.471" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:45.972" grpc.time_ms=0.79 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1883gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses1884gitlab # [ 203.246576] gitaly[1389]: time="2026-07-05T10:25:47.124Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0417E9G230C34K1FE18Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:17.021" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:47.122" grpc.time_ms=1.235 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab # [ 203.285882] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1141886gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses, in 0.19 seconds)1887[{'id': 3, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-05T10:25:44.040Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1888gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses1889gitlab # [ 204.491349] gitaly[1389]: time="2026-07-05T10:25:48.368Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0561TF1X4ZSNF5WVSGAF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:18.265" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:48.367" grpc.time_ms=1.343 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab # [ 204.531804] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1831891gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses, in 0.25 seconds)1892[{'id': 3, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-05T10:25:44.040Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1893gitlab # [ 204.874392] nix-daemon[3472]: accepted connection from pid 3510, user nixbot1894gitlab # [ 204.914211] gitaly[1389]: time="2026-07-05T10:25:48.792Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:18.289" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:48.790" grpc.time_ms=1.505 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1895gitlab # [ 205.020142] gitaly[1389]: time="2026-07-05T10:25:48.897Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:58.172" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:48.883" grpc.time_ms=14.315 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1896gitlab # [ 205.117534] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:48.993Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ff835288b65e0a1c5c7af50f","created_at":"2026-07-05T10:25:48.984Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-f698dce2ed614e34","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:48.990Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ff835288b65e0a1c5c7af50f: start","job_status":"start","queue_duration_s":0.003123,"scheduling_latency_s":0.003201}1897gitlab # [ 205.193789] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a7f9a657ca268f3e3bb99f4a038559b3eee13800 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 3591898gitlab # [ 205.205686] nixbot[3043]: {"ts": "2026-07-05T10:25:49.082Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a7f9a657ca268f3e3bb99f4a038559b3eee13800 \"HTTP/1.1 201 Created\""}1899gitlab # [ 205.284362] gitaly[1389]: time="2026-07-05T10:25:49.162Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX06123APQMKSKJTN5YBSW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:19.160" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:49.160" grpc.time_ms=0.916 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 205.318942] gitaly[1389]: time="2026-07-05T10:25:49.196Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:49.196" grpc.time_ms=0.621 pid=1389 span.kind=server system=grpc1901gitlab # [ 205.351123] gitaly[1389]: time="2026-07-05T10:25:49.229Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:49.228" grpc.time_ms=0.407 pid=1389 span.kind=server system=grpc1902gitlab # [ 205.379707] gitaly[1389]: time="2026-07-05T10:25:49.257Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:49.257" grpc.time_ms=0.476 pid=1389 span.kind=server system=grpc1903gitlab # [ 205.395995] gitaly[1389]: time="2026-07-05T10:25:49.268Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRX06123APQMKSKJTN5YBSW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:25:59.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:49.260" grpc.time_ms=7.748 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904gitlab # [ 205.419495] gitaly[1389]: time="2026-07-05T10:25:49.282Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF diskcache=f8d8139d-9f68-4d00-be03-09dd124a656b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:19.160" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:49.161" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 205.421035] gitaly[1389]: time="2026-07-05T10:25:49.282Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=106 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7328 command.oublock=0 command.real_time_ms=112 command.spawn_token_fork_ms=2 command.system_time_ms=71 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:19.160" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:49.161" grpc.time_ms=121.213 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1906gitlab # [ 205.435572] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a7f9a657ca268f3e3bb99f4a038559b3eee13800 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2221907gitlab # [ 205.436695] nixbot[3043]: {"ts": "2026-07-05T10:25:49.314Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a7f9a657ca268f3e3bb99f4a038559b3eee13800 \"HTTP/1.1 201 Created\""}1908gitlab # [ 205.498878] gitaly[1389]: time="2026-07-05T10:25:49.375Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-05T10:26:19.373" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:49.374" grpc.time_ms=1.614 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909gitlab # [ 205.525762] gitaly[1389]: time="2026-07-05T10:25:49.403Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:19.402" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:49.402" grpc.time_ms=0.858 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses1911gitlab # [ 205.584526] gitaly[1389]: time="2026-07-05T10:25:49.462Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:19.459" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:49.460" grpc.time_ms=1.318 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 205.602134] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.478Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5953f6029e12944a5d2ba77d","created_at":"2026-07-05T10:25:49.474Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"ff835288b65e0a1c5c7af50f","job_size_bytes":3,"pid":2102,"job_status":"deduplicated","message":"PipelineProcessWorker JID-5953f6029e12944a5d2ba77d: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1913gitlab # [ 205.605201] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.482Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ff835288b65e0a1c5c7af50f","created_at":"2026-07-05T10:25:48.984Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-f698dce2ed614e34","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:48.990Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2102,"job_status":"rescheduled","message":"PipelineProcessWorker JID-ff835288b65e0a1c5c7af50f: rescheduled"}1914gitlab # [ 205.618132] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.495Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ff835288b65e0a1c5c7af50f","created_at":"2026-07-05T10:25:48.984Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-f698dce2ed614e34","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:48.990Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ff835288b65e0a1c5c7af50f: done: 0.502118 sec","job_status":"done","queue_duration_s":0.003123,"scheduling_latency_s":0.003201,"gitaly_calls":4,"gitaly_duration_s":0.138427,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.082161,"redis_read_bytes":262,"redis_write_bytes":23387,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002351,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.005763,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001973,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.00715,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14468,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.057891,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000609,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000599,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003474,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.171445,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.382608683,"duration_s":0.502118,"completed_at":"2026-07-05T10:25:49.495Z","load_balancing_strategy":"primary","db_duration_s":0.060529,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1915gitlab # [ 205.700209] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.502Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"144b5cade9858424ba8e85d9","created_at":"2026-07-05T10:25:33.602Z","trace_propagation_headers":{"sentry-trace":"4b6004f287894231b86ea11fdeacd65b-9e9803d180354372","baggage":"sentry-trace_id=4b6004f287894231b86ea11fdeacd65b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZPR3J535M4A5MHJ6M1PW","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:33a43b88deca7a65468c683c657ddf0e6f689675ea83e41780108b0e80817df3","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.219Z","retry_count":0,"job_size_bytes":451,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-144b5cade9858424ba8e85d9: start","job_status":"start","queue_duration_s":0.282447,"scheduling_latency_s":0.282475}1916gitlab # [ 205.701788] sidekiq-cluster[2102]: {"severity":"WARN","time":"2026-07-05T10:25:49.540Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"144b5cade9858424ba8e85d9","created_at":"2026-07-05T10:25:33.602Z","trace_propagation_headers":{"sentry-trace":"4b6004f287894231b86ea11fdeacd65b-9e9803d180354372","baggage":"sentry-trace_id=4b6004f287894231b86ea11fdeacd65b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRWZPR3J535M4A5MHJ6M1PW","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:33a43b88deca7a65468c683c657ddf0e6f689675ea83e41780108b0e80817df3","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.219Z","retry_count":0,"job_size_bytes":451,"pid":2102,"sidekiq_tid":"m5j6","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-144b5cade9858424ba8e85d9: fail: 0.038569 sec","job_status":"fail","queue_duration_s":0.282447,"scheduling_latency_s":0.282475,"redis_calls":7,"redis_duration_s":0.00234,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001561,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038569,"completed_at":"2026-07-05T10:25:49.540Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.004541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1917gitlab # [ 206.333660] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.544Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9fead24f0b662781f851a458","created_at":"2026-07-05T10:25:49.152Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.154Z","job_size_bytes":23,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9fead24f0b662781f851a458: start","job_status":"start","queue_duration_s":0.390113,"scheduling_latency_s":0.390158}1918gitlab # [ 206.334816] gitaly[1389]: time="2026-07-05T10:25:49.507Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX06CFHBBTQBQ1TP04AGCW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:19.005" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:49.506" grpc.time_ms=0.538 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 206.351554] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 811920gitlab # [ 206.355764] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.632Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9fead24f0b662781f851a458","created_at":"2026-07-05T10:25:49.152Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.154Z","job_size_bytes":23,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9fead24f0b662781f851a458: done: 0.088354 sec","job_status":"done","queue_duration_s":0.390113,"scheduling_latency_s":0.390158,"redis_calls":7,"redis_duration_s":0.002019,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001488,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.088354,"completed_at":"2026-07-05T10:25:49.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.058505,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1921gitlab # [ 206.426342] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.634Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2b84ab9c3d87861099609218","created_at":"2026-07-05T10:25:49.287Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.290Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2b84ab9c3d87861099609218: start","job_status":"start","queue_duration_s":0.344301,"scheduling_latency_s":0.34433}1922gitlab # [ 206.513510] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.653Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2b84ab9c3d87861099609218","created_at":"2026-07-05T10:25:49.287Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.290Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2b84ab9c3d87861099609218: done: 0.019615 sec","job_status":"done","queue_duration_s":0.344301,"scheduling_latency_s":0.34433,"redis_calls":7,"redis_duration_s":0.00166,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019615,"completed_at":"2026-07-05T10:25:49.653Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002864,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1923gitlab # [ 206.518167] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.655Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"286f7b92529398b7212fc417","created_at":"2026-07-05T10:25:49.296Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.344Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-286f7b92529398b7212fc417: start","job_status":"start","queue_duration_s":0.310807,"scheduling_latency_s":0.31083}1924gitlab # [ 206.602310] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.681Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"286f7b92529398b7212fc417","created_at":"2026-07-05T10:25:49.296Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.344Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-286f7b92529398b7212fc417: done: 0.026017 sec","job_status":"done","queue_duration_s":0.310807,"scheduling_latency_s":0.31083,"redis_calls":7,"redis_duration_s":0.001941,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00138,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000561,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026017,"completed_at":"2026-07-05T10:25:49.681Z","load_balancing_strategy":"primary","db_duration_s":0.004437,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1925gitlab # [ 206.675160] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.683Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"f2307f21b1134a728d68d87b","created_at":"2026-07-05T10:25:49.353Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.356Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f2307f21b1134a728d68d87b: start","job_status":"start","queue_duration_s":0.327406,"scheduling_latency_s":0.327442}1926gitlab # [ 206.676400] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.711Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"f2307f21b1134a728d68d87b","created_at":"2026-07-05T10:25:49.353Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.356Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f2307f21b1134a728d68d87b: done: 0.028094 sec","job_status":"done","queue_duration_s":0.327406,"scheduling_latency_s":0.327442,"redis_calls":10,"redis_duration_s":0.00236,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000656,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028094,"completed_at":"2026-07-05T10:25:49.711Z","load_balancing_strategy":"primary","db_duration_s":0.003762,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1927gitlab # [ 206.749314] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.713Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"75be06271487af4047205feb","created_at":"2026-07-05T10:25:49.356Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.358Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-75be06271487af4047205feb: start","job_status":"start","queue_duration_s":0.354662,"scheduling_latency_s":0.354687}1928gitlab # [ 206.750601] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.732Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"75be06271487af4047205feb","created_at":"2026-07-05T10:25:49.356Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.358Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-75be06271487af4047205feb: done: 0.019019 sec","job_status":"done","queue_duration_s":0.354662,"scheduling_latency_s":0.354687,"redis_calls":7,"redis_duration_s":0.0016060000000000002,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019019,"completed_at":"2026-07-05T10:25:49.732Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1929gitlab # [ 206.857355] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.733Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8987b439b29ff01184443822","created_at":"2026-07-05T10:25:49.359Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.361Z","job_size_bytes":19,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8987b439b29ff01184443822: start","job_status":"start","queue_duration_s":0.371944,"scheduling_latency_s":0.371965}1930gitlab # [ 206.858933] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.752Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8987b439b29ff01184443822","created_at":"2026-07-05T10:25:49.359Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.361Z","job_size_bytes":19,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8987b439b29ff01184443822: done: 0.019103 sec","job_status":"done","queue_duration_s":0.371944,"scheduling_latency_s":0.371965,"redis_calls":8,"redis_duration_s":0.0017640000000000002,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001347,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019103,"completed_at":"2026-07-05T10:25:49.752Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1931gitlab # [ 206.944277] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.754Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"80b96d202daab2195a898377","created_at":"2026-07-05T10:25:49.362Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.363Z","job_size_bytes":66,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-80b96d202daab2195a898377: start","job_status":"start","queue_duration_s":0.390364,"scheduling_latency_s":0.390393}1932gitlab # [ 206.945844] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.769Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"80b96d202daab2195a898377","created_at":"2026-07-05T10:25:49.362Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.363Z","job_size_bytes":66,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-80b96d202daab2195a898377: done: 0.015741 sec","job_status":"done","queue_duration_s":0.390364,"scheduling_latency_s":0.390393,"redis_calls":7,"redis_duration_s":0.001817,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001168,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000649,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015741,"completed_at":"2026-07-05T10:25:49.769Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1933gitlab # [ 207.021409] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.771Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"1fc55e6db27d82955f66ac03","created_at":"2026-07-05T10:25:49.365Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.366Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1fc55e6db27d82955f66ac03: start","job_status":"start","queue_duration_s":0.404395,"scheduling_latency_s":0.404415}1934gitlab # [ 207.022966] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.796Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"1fc55e6db27d82955f66ac03","created_at":"2026-07-05T10:25:49.365Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.366Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1fc55e6db27d82955f66ac03: done: 0.025314 sec","job_status":"done","queue_duration_s":0.404395,"scheduling_latency_s":0.404415,"redis_calls":7,"redis_duration_s":0.002124,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000694,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025314,"completed_at":"2026-07-05T10:25:49.796Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1935gitlab # [ 207.095947] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.798Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"09b231c4b92a29554bf3bb6b","created_at":"2026-07-05T10:25:49.410Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.411Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-09b231c4b92a29554bf3bb6b: start","job_status":"start","queue_duration_s":0.386741,"scheduling_latency_s":0.38677}1936gitlab # [ 207.169366] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.820Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"09b231c4b92a29554bf3bb6b","created_at":"2026-07-05T10:25:49.410Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.411Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-09b231c4b92a29554bf3bb6b: done: 0.022104 sec","job_status":"done","queue_duration_s":0.386741,"scheduling_latency_s":0.38677,"redis_calls":7,"redis_duration_s":0.001973,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001522,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022104,"completed_at":"2026-07-05T10:25:49.820Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1937gitlab # [ 207.173269] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.821Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c9b93303068df766a4362ee8","created_at":"2026-07-05T10:25:49.412Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.413Z","job_size_bytes":6,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c9b93303068df766a4362ee8: start","job_status":"start","queue_duration_s":0.408153,"scheduling_latency_s":0.408175}1938gitlab # [ 207.252638] gitaly[1389]: time="2026-07-05T10:25:49.935Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-05T10:26:19.034" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:49.934" grpc.time_ms=0.92 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1939gitlab # [ 207.253769] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.834Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c9b93303068df766a4362ee8","created_at":"2026-07-05T10:25:49.412Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.413Z","job_size_bytes":6,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c9b93303068df766a4362ee8: done: 0.012981 sec","job_status":"done","queue_duration_s":0.408153,"scheduling_latency_s":0.408175,"redis_calls":7,"redis_duration_s":0.001536,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012981,"completed_at":"2026-07-05T10:25:49.834Z","load_balancing_strategy":"primary","db_duration_s":0.000444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1940gitlab # [ 207.354410] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.836Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"3ec1b75606feb7849923d0f7","created_at":"2026-07-05T10:25:49.414Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.419Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3ec1b75606feb7849923d0f7: start","job_status":"start","queue_duration_s":0.417593,"scheduling_latency_s":0.417614}1941gitlab # [ 207.355826] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.864Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"3ec1b75606feb7849923d0f7","created_at":"2026-07-05T10:25:49.414Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.419Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3ec1b75606feb7849923d0f7: done: 0.027842 sec","job_status":"done","queue_duration_s":0.417593,"scheduling_latency_s":0.417614,"redis_calls":10,"redis_duration_s":0.002454,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001512,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004048,"duration_s":0.027842,"completed_at":"2026-07-05T10:25:49.864Z","load_balancing_strategy":"primary","db_duration_s":0.004777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1942gitlab # [ 207.427288] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.867Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"c847c97535b3c0836c75cae0","created_at":"2026-07-05T10:25:49.482Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.486Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c847c97535b3c0836c75cae0: start","job_status":"start","queue_duration_s":0.380941,"scheduling_latency_s":0.380989}1943gitlab # [ 207.428895] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.995Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"c847c97535b3c0836c75cae0","created_at":"2026-07-05T10:25:49.482Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-9312649041ae4d20","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.486Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c847c97535b3c0836c75cae0: done: 0.128721 sec","job_status":"done","queue_duration_s":0.380941,"scheduling_latency_s":0.380989,"gitaly_calls":3,"gitaly_duration_s":0.008921,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.00904,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001072,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002096,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000304,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002763,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00025,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000456,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087543,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.100138426,"duration_s":0.128721,"completed_at":"2026-07-05T10:25:49.995Z","load_balancing_strategy":"primary","db_duration_s":0.027893,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1944gitlab # [ 207.499418] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:49.997Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"31fb4c1e2473412205e3b8b8","created_at":"2026-07-05T10:25:49.921Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-e3138a7f0dac4292","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.923Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-31fb4c1e2473412205e3b8b8: start","job_status":"start","queue_duration_s":0.073599,"scheduling_latency_s":0.073629}1945gitlab # [ 207.571140] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:50.024Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"31fb4c1e2473412205e3b8b8","created_at":"2026-07-05T10:25:49.921Z","trace_propagation_headers":{"sentry-trace":"49fe2bccdddd48bf881f797fad244c07-e3138a7f0dac4292","baggage":"sentry-trace_id=49fe2bccdddd48bf881f797fad244c07,sentry-release=Unknown"},"correlation_id":"01KWRX05N67RV5Y1WYSJFN3ZJF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:49.923Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-31fb4c1e2473412205e3b8b8: done: 0.02684 sec","job_status":"done","queue_duration_s":0.073599,"scheduling_latency_s":0.073629,"redis_calls":7,"redis_duration_s":0.001795,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001254,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000541,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02684,"completed_at":"2026-07-05T10:25:50.024Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1946gitlab # [ 207.645361] gitaly[1389]: time="2026-07-05T10:25:49.960Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:19.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:49.959" grpc.time_ms=0.648 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1947gitlab # [ 207.646875] gitaly[1389]: time="2026-07-05T10:25:49.983Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX05N67RV5Y1WYSJFN3ZJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:19.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:49.982" grpc.time_ms=0.586 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1948gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses, in 2.16 seconds)1949[{'id': 3, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-05T10:25:44.040Z', 'started_at': None, 'finished_at': '2026-07-05T10:25:48.969Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-05T10:25:49.287Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1950gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses1951gitlab # [ 208.823284] gitaly[1389]: time="2026-07-05T10:25:52.700Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX09FT1QMXTJWYFRDQ3E9P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:22.198" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:52.698" grpc.time_ms=1.467 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1952gitlab # [ 208.915514] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1581953gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses, in 0.22 seconds)1954[{'id': 3, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-05T10:25:44.040Z', 'started_at': None, 'finished_at': '2026-07-05T10:25:48.969Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-05T10:25:49.287Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1955gitlab # [ 209.340410] nix-daemon[3472]: accepted connection from pid 3575, user nixbot1956gitlab # [ 209.448404] gitaly[1389]: time="2026-07-05T10:25:53.326Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:23.323" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:53.324" grpc.time_ms=1.391 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957gitlab # [ 209.515275] gitaly[1389]: time="2026-07-05T10:25:53.393Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-05T10:26:03.038" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:53.379" grpc.time_ms=13.789 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1958gitlab # [ 209.609368] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-07-05T10:25:53.485Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"24f1b47b431b7b3a541d6e3f","created_at":"2026-07-05T10:25:53.473Z","trace_propagation_headers":{"sentry-trace":"51182a6c690245c2b662562887cbb18d-23783c5fc64a4a04","baggage":"sentry-trace_id=51182a6c690245c2b662562887cbb18d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWRX0A37HDBQDJTB7HJT8NJN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-05T10:25:53.478Z","job_size_bytes":3,"pid":2102,"sidekiq_tid":"m5zm","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-24f1b47b431b7b3a541d6e3f: start","job_status":"start","queue_duration_s":0.006602,"scheduling_latency_s":0.006669}1959gitlab # [ 209.661368] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a7f9a657ca268f3e3bb99f4a038559b3eee13800 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2441960gitlab # [ 209.664199] nixbot[3043]: {"ts": "2026-07-05T10:25:53.502Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a7f9a657ca268f3e3bb99f4a038559b3eee13800 \"HTTP/1.1 201 Created\""}1961gitlab # [ 209.763419] gitaly[1389]: time="2026-07-05T10:25:53.641Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:53.640" grpc.time_ms=0.686 pid=1389 span.kind=server system=grpc1962gitlab # [ 209.811969] gitaly[1389]: time="2026-07-05T10:25:53.689Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:53.689" grpc.time_ms=0.667 pid=1389 span.kind=server system=grpc1963gitlab # [ 209.845432] nix-daemon[3472]: accepted connection from pid 3600, user nixbot1964gitlab # [ 209.855648] gitaly[1389]: time="2026-07-05T10:25:53.733Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-05T10:25:53.732" grpc.time_ms=0.555 pid=1389 span.kind=server system=grpc1965gitlab # [ 209.874728] gitaly[1389]: time="2026-07-05T10:25:53.752Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:23.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:53.599" operations="[repack_refs]" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966gitlab # [ 209.876324] gitaly[1389]: time="2026-07-05T10:25:53.754Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:23.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:53.599" pid=1389 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-07-05 10:25:34.95684156 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root1967gitlab # [ 209.890988] gitaly[1389]: time="2026-07-05T10:25:53.769Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:23.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:53.599" pid=1389 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root1968gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses1969gitlab # [ 209.892304] gitaly[1389]: time="2026-07-05T10:25:53.769Z" level=info msg="optimized repository" clean_stale_data_time=13.799519ms clean_work_trees_time="164.266µs" commit_graph_time="7.823µs" correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.381µs" pid=1389 prune_time="7.822µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.101µs" storage_name=default system=housekeeping total_time=14.208231ms1970gitlab # [ 209.893151] gitaly[1389]: time="2026-07-05T10:25:53.769Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:23.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:53.599" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1971gitlab # [ 209.964942] gitaly[1389]: time="2026-07-05T10:25:53.842Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN diskcache=3f949002-9b9f-4d9d-81a1-b3365f0c899b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:23.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:53.599" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972gitlab # [ 209.981417] gitaly[1389]: time="2026-07-05T10:25:53.859Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=131 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=8925 command.oublock=0 command.real_time_ms=161 command.spawn_token_fork_ms=1 command.system_time_ms=85 command.user_time_ms=46 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0A37HDBQDJTB7HJT8NJN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-05T10:26:23.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-05T10:25:53.599" grpc.time_ms=244.049 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1973gitlab # [ 210.010088] gitaly[1389]: time="2026-07-05T10:25:53.887Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWRX0ANSNCTVZ4YY6CY59HMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-05T10:26:23.386" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-05T10:25:53.886" grpc.time_ms=0.824 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses, in 0.11 seconds)1975[{'id': 3, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-05T10:25:44.040Z', 'started_at': None, 'finished_at': '2026-07-05T10:25:48.969Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'a7f9a657ca268f3e3bb99f4a038559b3eee13800', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-05T10:25:49.287Z', 'started_at': None, 'finished_at': '2026-07-05T10:25:53.460Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]1976gitlab # [ 210.038550] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/05:10:25:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a7f9a657ca268f3e3bb99f4a038559b3eee13800/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 651977(finished: subtest: eval and build post commit statuses, in 10.41 seconds)1978(finished: run the VM test script, in 245.66 seconds)1979test script finished in 245.69s1980cleanup1981kill QemuMachine (pid 13)1982gitlab # [ 210.060819] gitaly[1389]: time="2026-07-05T10:25:53.938Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWRX0AP2GVE499Z9K3XA9Z96 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=rqemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)1983gitlab # o1984(finished: cleanup, in 1.29 seconds)