vm-test-run-nixbot-gitlab
x86_64-linux.nixbot-gitlab
· build #131
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 13)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.aHmjY6s35B', fmt=raw size=107374182412gitlab # mke2fs 1.47.3 (8-Jul-2025)13gitlab # Discarding device blocks: 0/262144 done14(finished: start all VMs, in 47.40 seconds)15gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes16gitlab: waiting for unit setup-gitlab.service17gitlab # Filesystem UUID: bab2b030-ad85-4218-8e00-0d421fc6cb7118gitlab: waiting for the VM to finish booting19gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8pw0dms91p54afz9cnbbxwzjm7p20wma-nixos-system-gitlab-test/init regInfo=/nix/store/x1lax7malg3g743vm5bx5mrwkp7gpzq3-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 488469758 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.006 MHz processor67gitlab # [ 0.000803] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000834] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000836] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000874] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002720] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002730] Using GB pages for direct mapping73gitlab # [ 0.002848] RAMDISK: [mem 0xbe4e0000-0xbffcffff]74gitlab # [ 0.002855] ACPI: Early table checksum verification disabled75gitlab # [ 0.002858] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002862] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002866] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002873] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002875] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002876] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002878] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002879] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002881] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002882] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002883] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002883] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002883] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002884] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003106] No NUMA configuration found90gitlab # [ 0.003107] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003109] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]92gitlab # [ 0.005420] Zone ranges:93gitlab # [ 0.005421] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005423] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005424] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005425] Device empty97gitlab # [ 0.005425] Movable zone start for each node98gitlab # [ 0.005426] Early memory node ranges99gitlab # [ 0.005427] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005428] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005428] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005430] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005449] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005717] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.115914] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.116339] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.116353] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.116383] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.116386] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.116388] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.116389] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.116390] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.116391] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.116393] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.116394] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.116399] TSC deadline timer available117gitlab # [ 0.116404] CPU topo: Max. logical packages: 1118gitlab # [ 0.116405] CPU topo: Max. logical dies: 1119gitlab # [ 0.116405] CPU topo: Max. dies per package: 1120gitlab # [ 0.116408] CPU topo: Max. threads per core: 1121gitlab # [ 0.116409] CPU topo: Num. cores per package: 4122gitlab # [ 0.116409] CPU topo: Num. threads per package: 4123gitlab # [ 0.116410] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.116433] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.116450] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.116453] kvm-guest: setup PV sched yield127gitlab # [ 0.116468] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.116469] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.116470] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.116472] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.116473] Booting paravirtualized kernel on KVM132gitlab # [ 0.116477] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.120933] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.123175] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.123223] kvm-guest: PV spinlocks enabled136gitlab # [ 0.123225] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.123228] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8pw0dms91p54afz9cnbbxwzjm7p20wma-nixos-system-gitlab-test/init regInfo=/nix/store/x1lax7malg3g743vm5bx5mrwkp7gpzq3-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.123323] Unknown kernel command line parameters "regInfo=/nix/store/x1lax7malg3g743vm5bx5mrwkp7gpzq3-closure-info/registration", will be passed to user space.139gitlab # [ 0.123336] random: crng init done140gitlab # [ 0.123336] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.132333] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.136704] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.136785] software IO TLB: area num 4.144gitlab # [ 0.212950] Fallback order for Node 0: 0145gitlab # [ 0.212958] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.212960] Policy zone: Normal147gitlab # [ 0.215509] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.225100] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.239322] allocated 12582912 bytes of page_ext150gitlab # [ 0.249314] ftrace: allocating 48565 entries in 192 pages151gitlab # [ 0.249316] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.250170] Dynamic Preempt: lazy153gitlab # [ 0.250341] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.250342] rcu: RCU event tracing is enabled.155gitlab # [ 0.250343] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.250344] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.250345] Rude variant of Tasks RCU enabled.158gitlab # [ 0.250345] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.250346] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.250347] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.250372] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.250374] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.250375] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.254720] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.254999] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.255114] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)167gitlab # [ 0.258632] Console: colour VGA+ 80x25168gitlab # [ 0.258634] printk: legacy console [tty0] enabled169gitlab # [ 0.289195] printk: legacy console [ttyS0] enabled170gitlab # [ 0.389494] ACPI: Core revision 20250807171gitlab # [ 0.390350] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns172gitlab # [ 0.391937] APIC: Switch to symmetric I/O mode setup173gitlab # [ 0.392916] x2apic enabled174gitlab # [ 0.393681] APIC: Switched APIC routing to: physical x2apic175gitlab # [ 0.394607] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()176gitlab # [ 0.395825] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()177gitlab # [ 0.397286] kvm-guest: setup PV IPIs178gitlab # [ 0.398845] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1179gitlab # [ 0.399815] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983d4fda2, max_idle_ns: 440795302975 ns180gitlab # [ 0.401522] Calibrating delay loop (skipped) preset value.. 4800.01 BogoMIPS (lpj=2400006)181gitlab # [ 0.403601] x86/cpu: User Mode Instruction Prevention (UMIP) activated182gitlab # [ 0.404689] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127183gitlab # [ 0.405520] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0184gitlab # [ 0.406523] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto185gitlab # [ 0.408520] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl186gitlab # [ 0.410520] Transient Scheduler Attacks: Vulnerable: No microcode187gitlab # [ 0.411519] Spectre V2 : Mitigation: Enhanced / Automatic IBRS188gitlab # [ 0.412519] Speculative Return Stack Overflow: Mitigation: Safe RET189gitlab # [ 0.413519] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization190gitlab # [ 0.414524] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier191gitlab # [ 0.415520] active return thunk: srso_alias_return_thunk192gitlab # [ 0.417185] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'193gitlab # [ 0.418519] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'194gitlab # [ 0.419519] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'195gitlab # [ 0.420519] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'196gitlab # [ 0.421519] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'197gitlab # [ 0.422519] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'198gitlab # [ 0.423519] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'199gitlab # [ 0.424519] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256200gitlab # [ 0.425519] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64201gitlab # [ 0.426519] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512202gitlab # [ 0.427519] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024203gitlab # [ 0.428519] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8204gitlab # [ 0.429519] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.205gitlab # [ 0.459657] Freeing SMP alternatives memory: 44K206gitlab # [ 0.460476] pid_max: default: 32768 minimum: 301207gitlab # [ 0.461206] LSM: initializing lsm=capability,landlock,yama,bpf,ima208gitlab # [ 0.462399] landlock: Up and running.209gitlab # [ 0.462983] Yama: becoming mindful.210gitlab # [ 0.463729] LSM support for eBPF active211gitlab # [ 0.464748] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)212gitlab # [ 0.465667] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)213gitlab # [ 0.468817] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)214gitlab # [ 0.470116] Performance Events: Fam17h+ core perfctr, AMD PMU driver.215gitlab # [ 0.470528] ... version: 2216gitlab # [ 0.471275] ... bit width: 48217gitlab # [ 0.471522] ... generic counters: 6218gitlab # [ 0.472257] ... generic bitmap: 000000000000003f219gitlab # [ 0.472521] ... fixed-purpose counters: 0220gitlab # [ 0.473268] ... fixed-purpose bitmap: 0000000000000000221gitlab # [ 0.473521] ... value mask: 0000ffffffffffff222gitlab # [ 0.474451] ... max period: 00007fffffffffff223gitlab # [ 0.475237] ... global_ctrl mask: 000000000000003f224gitlab # [ 0.475623] signal: max sigframe size: 3376225gitlab # [ 0.476489] rcu: Hierarchical SRCU implementation.226gitlab # [ 0.477125] rcu: Max phase no-delay instances is 400.227gitlab # [ 0.477724] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level228gitlab # [ 0.482922] smp: Bringing up secondary CPUs ...229gitlab # [ 0.483947] smpboot: x86: Booting SMP configuration:230gitlab # [ 0.484524] .... node #0, CPUs: #1 #2 #3231gitlab # [ 0.487662] smp: Brought up 1 node, 4 CPUs232gitlab # [ 0.489253] smpboot: Total of 4 processors activated (19200.04 BogoMIPS)233gitlab # [ 0.489854] Memory: 6013204K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258296K reserved, 0K cma-reserved)234gitlab # [ 0.492522] devtmpfs: initialized235gitlab # [ 0.493526] x86/mm: Memory block size: 128MB236gitlab # [ 0.496663] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns237gitlab # [ 0.497568] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)238gitlab # [ 0.498595] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).239gitlab # [ 0.499641] pinctrl core: initialized pinctrl subsystem240gitlab # [ 0.500774] PM: RTC time: 06:33:41, date: 2026-07-21241gitlab # [ 0.504561] NET: Registered PF_NETLINK/PF_ROUTE protocol family242gitlab # [ 0.506840] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations243gitlab # [ 0.507580] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations244gitlab # [ 0.509641] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations245gitlab # [ 0.510533] audit: initializing netlink subsys (disabled)246gitlab # [ 0.511579] audit: type=2000 audit(1784615620.956:1): state=initialized audit_enabled=0 res=1247gitlab # [ 0.511828] thermal_sys: Registered thermal governor 'fair_share'248gitlab # [ 0.513522] thermal_sys: Registered thermal governor 'bang_bang'249gitlab # [ 0.514522] thermal_sys: Registered thermal governor 'step_wise'250gitlab # [ 0.515512] thermal_sys: Registered thermal governor 'user_space'251gitlab # [ 0.516299] thermal_sys: Registered thermal governor 'power_allocator'252gitlab # [ 0.516534] cpuidle: using governor menu253gitlab # [ 0.518778] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5254gitlab # [ 0.519764] PCI: Using configuration type 1 for base access255gitlab # [ 0.520523] PCI: Using configuration type 1 for extended access256gitlab # [ 0.521695] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.257gitlab # [ 0.523770] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages258gitlab # [ 0.525438] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page259gitlab # [ 0.526412] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages260gitlab # [ 0.527396] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page261gitlab # [ 0.533221] ACPI: Added _OSI(Module Device)262gitlab # [ 0.534528] ACPI: Added _OSI(Processor Device)263gitlab # [ 0.536528] ACPI: Added _OSI(Processor Aggregator Device)264gitlab # [ 0.543386] ACPI: 1 ACPI AML tables successfully acquired and loaded265gitlab # [ 0.550829] ACPI: Interpreter enabled266gitlab # [ 0.552597] ACPI: PM: (supports S0 S3 S4 S5)267gitlab # [ 0.554530] ACPI: Using IOAPIC for interrupt routing268gitlab # [ 0.556590] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug269gitlab # [ 0.559530] PCI: Using E820 reservations for host bridge windows270gitlab # [ 0.562862] ACPI: Enabled 2 GPEs in block 00 to 0F271gitlab # [ 0.575432] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])272gitlab # [ 0.577572] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]273gitlab # [ 0.581306] acpiphp: Slot [3] registered274gitlab # [ 0.582553] acpiphp: Slot [4] registered275gitlab # [ 0.583565] acpiphp: Slot [5] registered276gitlab # [ 0.585583] acpiphp: Slot [6] registered277gitlab # [ 0.586566] acpiphp: Slot [7] registered278gitlab # [ 0.587566] acpiphp: Slot [8] registered279gitlab # [ 0.589732] acpiphp: Slot [9] registered280gitlab # [ 0.590567] acpiphp: Slot [10] registered281gitlab # [ 0.592565] acpiphp: Slot [11] registered282gitlab # [ 0.593565] acpiphp: Slot [12] registered283gitlab # [ 0.595565] acpiphp: Slot [13] registered284gitlab # [ 0.596304] acpiphp: Slot [14] registered285gitlab # [ 0.596542] acpiphp: Slot [15] registered286gitlab # [ 0.597277] acpiphp: Slot [16] registered287gitlab # [ 0.598551] acpiphp: Slot [17] registered288gitlab # [ 0.599265] acpiphp: Slot [18] registered289gitlab # [ 0.599541] acpiphp: Slot [19] registered290gitlab # [ 0.600289] acpiphp: Slot [20] registered291gitlab # [ 0.600550] acpiphp: Slot [21] registered292gitlab # [ 0.601245] acpiphp: Slot [22] registered293gitlab # [ 0.602541] acpiphp: Slot [23] registered294gitlab # [ 0.603293] acpiphp: Slot [24] registered295gitlab # [ 0.603543] acpiphp: Slot [25] registered296gitlab # [ 0.604235] acpiphp: Slot [26] registered297gitlab # [ 0.605542] acpiphp: Slot [27] registered298gitlab # [ 0.606282] acpiphp: Slot [28] registered299gitlab # [ 0.606543] acpiphp: Slot [29] registered300gitlab # [ 0.607243] acpiphp: Slot [30] registered301gitlab # [ 0.608542] acpiphp: Slot [31] registered302gitlab # [ 0.609272] PCI host bridge to bus 0000:00303gitlab # [ 0.609526] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]304gitlab # [ 0.610522] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]305gitlab # [ 0.611528] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]306gitlab # [ 0.612522] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]307gitlab # [ 0.614524] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]308gitlab # [ 0.615523] pci_bus 0000:00: root bus resource [bus 00-ff]309gitlab # [ 0.616682] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint310gitlab # [ 0.620528] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint311gitlab # [ 0.624600] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint312gitlab # [ 0.631614] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]313gitlab # [ 0.633597] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk314gitlab # [ 0.636529] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk315gitlab # [ 0.638528] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk316gitlab # [ 0.641527] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk317gitlab # [ 0.644994] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint318gitlab # [ 0.650611] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]319gitlab # [ 0.656160] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint320gitlab # [ 0.660231] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI321gitlab # [ 0.662544] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB322gitlab # [ 0.666059] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint323gitlab # [ 0.685609] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]324gitlab # [ 0.686535] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]325gitlab # [ 0.687533] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]326gitlab # [ 0.689024] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]327gitlab # [ 0.691553] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint328gitlab # [ 0.695553] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]329gitlab # [ 0.696474] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]330gitlab # [ 0.698311] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]331gitlab # [ 0.710478] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]332gitlab # [ 0.713967] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint333gitlab # [ 0.717405] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]334gitlab # [ 0.718223] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]335gitlab # [ 0.719544] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]336gitlab # [ 0.721804] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint337gitlab # [ 0.727192] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]338gitlab # [ 0.728529] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]339gitlab # [ 0.729527] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]340gitlab # [ 0.731776] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint341gitlab # [ 0.734553] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]342gitlab # [ 0.735440] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]343gitlab # [ 0.736307] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]344gitlab # [ 0.739770] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint345gitlab # [ 0.743530] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]346gitlab # [ 0.744420] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]347gitlab # [ 0.745273] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]348gitlab # [ 0.748006] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint349gitlab # [ 0.756560] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]350gitlab # [ 0.759545] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]351gitlab # [ 0.762582] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]352gitlab # [ 0.768754] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint353gitlab # [ 0.778575] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]354gitlab # [ 0.780544] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]355gitlab # [ 0.783583] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]356gitlab # [ 0.786542] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]357gitlab # [ 0.790985] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint358gitlab # [ 0.796563] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]359gitlab # [ 0.798569] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]360gitlab # [ 0.803019] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint361gitlab # [ 0.808560] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]362gitlab # [ 0.809457] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]363gitlab # [ 0.810305] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]364gitlab # [ 0.812875] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint365gitlab # [ 0.818563] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]366gitlab # [ 0.820544] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]367gitlab # [ 0.823581] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]368gitlab # [ 0.835022] ACPI: PCI: Interrupt link LNKA configured for IRQ 10369gitlab # [ 0.837854] ACPI: PCI: Interrupt link LNKB configured for IRQ 10370gitlab # [ 0.839816] ACPI: PCI: Interrupt link LNKC configured for IRQ 11371gitlab # [ 0.841788] ACPI: PCI: Interrupt link LNKD configured for IRQ 11372gitlab # [ 0.843676] ACPI: PCI: Interrupt link LNKS configured for IRQ 9373gitlab # [ 0.848873] iommu: Default domain type: Translated374gitlab # [ 0.849522] iommu: DMA domain TLB invalidation policy: lazy mode375gitlab # [ 0.851841] ACPI: bus type USB registered376gitlab # [ 0.852843] usbcore: registered new interface driver usbfs377gitlab # [ 0.855590] usbcore: registered new interface driver hub378gitlab # [ 0.857591] usbcore: registered new device driver usb379gitlab # [ 0.862756] NetLabel: Initializing380gitlab # [ 0.864528] NetLabel: domain hash size = 128381gitlab # [ 0.866526] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO382gitlab # [ 0.868648] NetLabel: unlabeled traffic allowed by default383gitlab # [ 0.870551] PCI: Using ACPI for IRQ routing384gitlab # [ 0.873654] pci 0000:00:02.0: vgaarb: setting as boot VGA device385gitlab # [ 0.874518] pci 0000:00:02.0: vgaarb: bridge control possible386gitlab # [ 0.874518] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none387gitlab # [ 0.881537] vgaarb: loaded388gitlab # [ 0.884053] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0389gitlab # [ 0.885526] hpet0: 3 comparators, 64-bit 100.000000 MHz counter390gitlab # [ 0.891871] clocksource: Switched to clocksource kvm-clock391gitlab # [ 0.897081] VFS: Disk quotas dquot_6.6.0392gitlab # [ 0.898872] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)393gitlab # [ 0.902212] pnp: PnP ACPI init394gitlab # [ 0.905023] pnp: PnP ACPI: found 6 devices395gitlab # [ 0.918070] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns396gitlab # [ 0.921897] clocksource: Switched to clocksource acpi_pm397gitlab # [ 0.924465] NET: Registered PF_INET protocol family398gitlab # [ 0.929444] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)399gitlab # [ 0.957065] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)400gitlab # [ 0.958574] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)401gitlab # [ 0.960028] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)402gitlab # [ 0.964553] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)403gitlab # [ 0.965907] TCP: Hash tables configured (established 65536 bind 65536)404gitlab # [ 0.967372] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)405gitlab # [ 0.968997] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)406gitlab # [ 0.970420] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.971828] NET: Registered PF_UNIX/PF_LOCAL protocol family408gitlab # [ 0.972848] NET: Registered PF_XDP protocol family409gitlab # [ 0.973715] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]410gitlab # [ 0.974768] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]411gitlab # [ 0.975803] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]412gitlab # [ 0.976928] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]413gitlab # [ 0.978073] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]414gitlab # [ 0.979438] pci 0000:00:01.0: PIIX3: Enabling Passive Release415gitlab # [ 0.980427] pci 0000:00:00.0: Limiting direct PCI/PCI transfers416gitlab # [ 0.982880] ACPI: \_SB_.LNKD: Enabled at IRQ 11417gitlab # [ 0.985598] PCI: CLS 0 bytes, default 64418gitlab # [ 0.986429] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)419gitlab # [ 0.987008] Trying to unpack rootfs image as initramfs...420gitlab # [ 0.987545] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)421gitlab # [ 0.987668] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983d4fda2, max_idle_ns: 440795302975 ns422gitlab # [ 1.010862] Initialise system trusted keyrings423gitlab # [ 1.012120] workingset: timestamp_bits=40 max_order=21 bucket_order=0424gitlab # [ 1.023608] Key type asymmetric registered425gitlab # [ 1.024363] Asymmetric key parser 'x509' registered426gitlab # [ 1.025317] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)427gitlab # [ 1.026943] io scheduler mq-deadline registered428gitlab # [ 1.027732] io scheduler kyber registered429gitlab # [ 1.031193] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled430gitlab # [ 1.032635] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A431gitlab # [ 1.035072] Linux agpgart interface v0.103432gitlab # [ 1.035849] ACPI: bus type drm_connector registered433gitlab # [ 1.038275] usbcore: registered new interface driver usbserial_generic434gitlab # [ 1.039408] usbserial: USB Serial support registered for generic435gitlab # [ 1.040427] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.436gitlab # [ 1.040427] Please enable it if your BIOS has the CPPC option.437gitlab # [ 1.042792] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled438gitlab # [ 1.044237] drop_monitor: Initializing network drop monitor service439gitlab # [ 1.045465] NET: Registered PF_INET6 protocol family440gitlab # [ 1.047356] Segment Routing with IPv6441gitlab # [ 1.048075] In-situ OAM (IOAM) with IPv6442gitlab # [ 1.050029] IPI shorthand broadcast: enabled443gitlab # [ 1.054414] sched_clock: Marking stable (915014421, 138512524)->(1074425083, -20898138)444gitlab # [ 1.056278] registered taskstats version 1445gitlab # [ 1.057335] Loading compiled-in X.509 certificates446gitlab # [ 1.067081] Demotion targets for Node 0: null447gitlab # [ 1.068212] Key type .fscrypt registered448gitlab # [ 1.068918] Key type fscrypt-provisioning registered449gitlab # [ 1.069913] ima: No TPM chip found, activating TPM-bypass!450gitlab # [ 1.070831] ima: Allocated hash algorithm: sha1451gitlab # [ 1.071673] ima: No architecture policies found452gitlab # [ 1.072830] PM: Magic number: 2:249:570453gitlab # [ 1.074455] RAS: Correctable Errors collector initialized.454gitlab # [ 1.079023] clk: Disabling unused clocks455gitlab # [ 1.079735] PM: genpd: Disabling unused power domains456gitlab # [ 1.268088] Freeing initrd memory: 27584K457gitlab # [ 1.275343] Freeing unused decrypted memory: 2028K458gitlab # [ 1.281711] Freeing unused kernel image (initmem) memory: 3644K459gitlab # [ 1.284364] Write protecting the kernel read-only data: 32768k460gitlab # [ 1.289387] Freeing unused kernel image (text/rodata gap) memory: 1332K461gitlab # [ 1.293376] Freeing unused kernel image (rodata/data gap) memory: 816K462gitlab # [ 1.360189] x86/mm: Checked W+X mappings: passed, no W+X pages found.463gitlab # [ 1.361346] Run /init as init process464gitlab # [ 1.373247] systemd[1]: Inserted module 'autofs4'465gitlab # [ 1.395028] fuse: init (API version 7.45)466gitlab # [ 1.403065] ACPI: \_SB_.LNKC: Enabled at IRQ 10467gitlab # [ 1.409573] ACPI: \_SB_.LNKA: Enabled at IRQ 10468gitlab # [ 1.412774] ACPI: \_SB_.LNKB: Enabled at IRQ 11469gitlab # [ 1.449322] systemd[1]: Successfully made /usr/ read-only.470gitlab # [ 1.786361] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)471gitlab # [ 1.798605] systemd[1]: Detected virtualization kvm.472gitlab # [ 1.829468] systemd[1]: Detected architecture x86-64.473gitlab # [ 1.831684] systemd[1]: Running in initrd.474gitlab # [ 1.834259] systemd[1]: Initializing machine ID from random generator.475gitlab # [ 1.837197] systemd[1]: Hostname set to <gitlab>.476gitlab # [ 1.900280] systemd[1]: Queued start job for default target Initrd Default Target.477gitlab # [ 2.000664] systemd[1]: Created slice Slice /system/modprobe.478gitlab # [ 2.001913] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.479gitlab # [ 2.003361] systemd[1]: Expecting device /dev/disk/by-label/nix-store...480gitlab # [ 2.004550] systemd[1]: Expecting device /dev/disk/by-label/nixos...481gitlab # [ 2.005681] systemd[1]: Reached target Path Units.482gitlab # [ 2.006567] systemd[1]: Reached target Slice Units.483gitlab # [ 2.007451] systemd[1]: Reached target Swaps.484gitlab # [ 2.008293] systemd[1]: Reached target Timer Units.485gitlab # [ 2.009292] systemd[1]: Listening on D-Bus System Message Bus Socket.486gitlab # [ 2.010532] systemd[1]: Listening on Journal Socket (/dev/log).487gitlab # [ 2.011687] systemd[1]: Listening on Journal Sockets.488gitlab # [ 2.012692] systemd[1]: Listening on udev Control Socket.489gitlab # [ 2.013712] systemd[1]: Listening on udev Kernel Socket.490gitlab # [ 2.014678] systemd[1]: Reached target Socket Units.491gitlab # [ 2.018949] systemd[1]: Starting Create List of Static Device Nodes...492gitlab # [ 2.022024] systemd[1]: Starting Load Kernel Module 9pnet_virtio...493gitlab # [ 2.025468] systemd[1]: Starting Load Kernel Module configfs...494gitlab # [ 2.034399] systemd[1]: Starting Journal Service...495gitlab # [ 2.040267] systemd[1]: Starting Load Kernel Modules...496gitlab # [ 2.041314] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki497gitlab # [ 2.053085] systemd[1]: Starting Coldplug All udev Devices...498gitlab # [ 2.060906] systemd-journald[150]: Collecting audit messages is disabled.499gitlab # [ 2.064325] netfs: FS-Cache loaded500gitlab # [ 2.071054] 9pnet: Installing 9P2000 support501gitlab # [ 2.072284] systemd[1]: Finished Create List of Static Device Nodes.502gitlab # [ 2.078833] systemd[1]: modprobe@configfs.service: Deactivated successfully.503gitlab # [ 2.083816] systemd[1]: Finished Load Kernel Module configfs.504gitlab # [ 2.090335] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.505gitlab # [ 2.090441] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.506gitlab # [ 2.094043] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev507gitlab # [ 2.095907] systemd[1]: Finished Load Kernel Module 9pnet_virtio.508gitlab # [ 2.100501] systemd[1]: Finished Load Kernel Modules.509gitlab # [ 2.101964] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config510gitlab # [ 2.105797] systemd[1]: Starting Apply Kernel Variables...511gitlab # [ 1.981429] systemd-modules-load[154]: Using 4 probe threads512gitlab # [ 2.120662] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...513gitlab # [ 1.982719] systemd-modules-load[154]: Inserted module 'virtio_balloon'514gitlab # [ 1.984473] systemd-modules-l[ 2.122960] systemd[1]: Started Journal Service.515gitlab # oad[154]: Inserted module 'virtio_gpu'516gitlab # [ 1.986063] systemd-modules-load[154]: Inserted module 'dm_mod'517gitlab # [ 1.989334] systemd[1]: Finished Apply Kernel Variables.518gitlab # [ 1.994299] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.519gitlab # [ 1.997320] systemd[1]: Starting Create Static Device Nodes in /dev...520gitlab # [ 2.016346] systemd[1]: Finished Create Static Device Nodes in /dev.521gitlab # [ 2.017267] systemd[1]: Reached target Preparation for Local File Systems.522gitlab # [ 2.018186] systemd[1]: Reached target Local File Systems.523gitlab # [ 2.018998] systemd[1]: Starting Create System Files and Directories...524gitlab # [ 2.032190] systemd[1]: Starting Rule-based Manager for Device Events and Files...525gitlab # [ 2.038228] systemd[1]: Finished Create System Files and Directories.526gitlab # [ 2.048425] systemd-udevd[191]: Using default interface naming scheme 'v260'.527gitlab # [ 2.059928] systemd[1]: Started Rule-based Manager for Device Events and Files.528gitlab # [ 2.077780] systemd[1]: Finished Coldplug All udev Devices.529gitlab # [ 2.078687] systemd[1]: Reached target System Initialization.530gitlab # [ 2.079530] systemd[1]: Reached target Basic System.531gitlab # [ 2.315484] uhci_hcd 0000:00:01.2: UHCI Host Controller532gitlab # [ 2.316619] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1533gitlab # [ 2.317814] uhci_hcd 0000:00:01.2: detected 2 ports534gitlab # [ 2.321373] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140535gitlab # [ 2.322500] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18536gitlab # [ 2.323788] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1537gitlab # [ 2.324856] usb usb1: Product: UHCI Host Controller538gitlab # [ 2.325624] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd539gitlab # [ 2.326572] usb usb1: SerialNumber: 0000:00:01.2540gitlab # [ 2.331634] hub 1-0:1.0: USB hub found541gitlab # [ 2.332296] hub 1-0:1.0: 2 ports detected542gitlab # [ 2.335768] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12543gitlab # [ 2.338001] SCSI subsystem initialized544gitlab # [ 2.349303] serio: i8042 KBD port at 0x60,0x64 irq 1545gitlab # [ 2.350234] serio: i8042 AUX port at 0x60,0x64 irq 12546gitlab # [ 2.355421] virtio_blk virtio4: 4/0/0 default/read/poll queues547gitlab # [ 2.364424] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)548gitlab # [ 2.266870] (udev-worker)[206]: Network interface NamePolicy= disabled on kernel command line.549gitlab # [ 2.408716] virtio_blk virtio5: 4/0/0 default/read/poll queues550gitlab # [ 2.412644] virtio_blk virtio5: [vdb] 11451064 512-byte logical blocks (5.86 GB/5.46 GiB)551gitlab # [ 2.412880] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0552gitlab # [ 2.285510] systemd[1]: Starting Virtual Console Setup...553gitlab # [ 2.286759] (udev-worker)[209]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.554gitlab # [ 2.290293] (udev-worker)[209]: Network interface NamePolicy= disabled on kernel command line.555gitlab # [ 2.430695] scsi host0: ata_piix556gitlab # [ 2.434081] scsi host1: ata_piix557gitlab # [ 2.434894] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0558gitlab # [ 2.436985] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0559gitlab # [ 2.300817] systemd-vconsole-setup[231]: Configuration of first virtual console was skipped, ignoring remaining ones.560gitlab # [ 2.302898] systemd[1]: Found device /dev/disk/by-label/nixos.561gitlab # [ 2.303916] systemd[1]: Reached target Initrd Root Device.562gitlab # [ 2.304795] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...563gitlab # [ 2.306407] systemd[1]: Finished Virtual Console Setup.564gitlab # [ 2.310180] systemd[1]: Found device /dev/disk/by-label/nix-store.565gitlab # [ 2.330579] systemd-fsck[243]: nixos: clean, 12/65536 files, 13019/262144 blocks566gitlab # [ 2.335861] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.567gitlab # [ 2.578703] usb 1-1: new full-speed USB device number 2 using uhci_hcd568gitlab # [ 2.595222] ata2: found unknown device (class 0)569gitlab # [ 2.598197] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100570gitlab # [ 2.602772] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5571gitlab # [ 2.664984] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray572gitlab # [ 2.692291] cdrom: Uniform CD-ROM driver Revision: 3.20573gitlab # [ 2.745933] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00574gitlab # [ 2.748999] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10575gitlab # [ 2.751690] usb 1-1: Product: QEMU USB Tablet576gitlab # [ 2.753351] usb 1-1: Manufacturer: QEMU577gitlab # [ 2.754860] usb 1-1: SerialNumber: 28754-0000:00:01.2-1578gitlab # [ 2.790871] hid: raw HID events driver (C) Jiri Kosina579gitlab # [ 2.803327] usbcore: registered new interface driver usbhid580gitlab # [ 2.805012] usbhid: USB HID core driver581gitlab # [ 2.813299] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2582gitlab # [ 2.816651] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0583gitlab # [ 2.934761] systemd[1]: Mounting /sysroot...584gitlab # [ 3.228642] EXT4-fs (vda): mounted filesystem bab2b030-ad85-4218-8e00-0d421fc6cb71 r/w with ordered data mode. Quota mode: none.585gitlab # [ 3.097444] systemd[1]: Mounted /sysroot.586gitlab # [ 3.099273] systemd[1]: Reached target Initrd Root File System.587gitlab # [ 3.105158] systemd[1]: Mounting /sysroot/nix/.ro-store...588gitlab # [ 3.109619] systemd[1]: Mounting /sysroot/nix/.rw-store...589gitlab # [ 3.114192] systemd[1]: Mounting /sysroot/run...590gitlab # [ 3.124201] systemd[1]: Mounting /sysroot/tmp/shared...591gitlab # [ 3.131322] systemd[1]: Mounting /sysroot/tmp/xchg...592gitlab # [ 3.138282] systemd[1]: Starting Mountpoints Configured in the Real Root...593gitlab # [ 3.288700] 9p: Installing v9fs 9p2000 file system support594gitlab # [ 3.151987] systemd[1]: Mounted /sysroot/nix/.rw-store.595gitlab # [ 3.154161] systemd[1]: Mounted /sysroot/run.596gitlab # [ 3.155924] systemd-sysroot-fstab-check[272]: /sysroot should be mounted in the initrd, will request daemon-reload.597gitlab # [ 3.161180] systemd[1]: Starting rw-sysroot-nix-store.service[ 3.300450] erofs (device vdb): mounted with root inode @ nid 128.598gitlab # ...599gitlab # [ 3.163841] systemd[1]: Mounted /sysroot/tmp/shared.600gitlab # [ 3.165189] systemd[1]: Mounted /sysroot/tmp/xchg.601gitlab # [ 3.166552] systemd[1]: Reload requested from client PID 272 ('systemd-sysroot') (unit initrd-parse-etc.service)...602gitlab # [ 3.168572] systemd[1]: Reloading...603gitlab # [ 3.378102] systemd[1]: Reloading finished in 213 ms.604gitlab # [ 3.432179] systemd-sysroot-fstab-check[272]: Requesting initrd-fs.target/start/replace...605gitlab # [ 3.531295] systemd[1]: Mounted /sysroot/nix/.ro-store.606gitlab # [ 3.533376] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.607gitlab # [ 3.537087] systemd[1]: Finished rw-sysroot-nix-store.service.608gitlab # [ 3.543307] systemd-sysroot-fstab-check[272]: Requesting swap.target/start/replace...609gitlab # [ 3.547457] systemd[1]: initrd-parse-etc.service: Deactivated successfully.610gitlab # [ 3.551063] systemd[1]: Finished Mountpoints Configured in the Real Root.611gitlab # [ 3.553313] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.612gitlab # [ 3.562280] systemd[1]: Starting rw-sysroot-nix-store.service...613gitlab # [ 3.578463] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.614gitlab # [ 3.584268] systemd[1]: Finished rw-sysroot-nix-store.service.615gitlab # [ 3.935270] systemd[1]: Mounting /sysroot/nix/store...616gitlab # [ 3.984559] systemd[1]: Mounted /sysroot/nix/store.617gitlab # [ 3.986681] systemd[1]: Reached target Initrd File Systems.618gitlab # [ 3.991852] systemd[1]: Starting Find NixOS closure...619gitlab # [ 3.996817] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...620gitlab # [ 4.024526] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.621gitlab # [ 4.027290] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.622gitlab # [ 4.040901] systemd[1]: Finished Find NixOS closure.623gitlab # [ 4.042763] systemd[1]: Reached target Initrd Default Target.624gitlab # [ 4.047542] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...625gitlab # [ 4.078769] systemd[1]: Stopped target Initrd Default Target.626gitlab # [ 4.080894] systemd[1]: Stopped target Basic System.627gitlab # [ 4.082640] systemd[1]: Stopped target Initrd Root Device.628gitlab # [ 4.084448] systemd[1]: Stopped target Path Units.629gitlab # [ 4.086116] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.630gitlab # [ 4.088527] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.631gitlab # [ 4.090986] systemd[1]: Stopped target Slice Units.632gitlab # [ 4.092683] systemd[1]: Stopped target Socket Units.633gitlab # [ 4.094374] systemd[1]: Stopped target System Initialization.634gitlab # [ 4.096351] systemd[1]: Stopped target Swaps.635gitlab # [ 4.097941] systemd[1]: Stopped target Timer Units.636gitlab # [ 4.099698] systemd[1]: dbus.socket: Deactivated successfully.637gitlab # [ 4.101702] systemd[1]: Closed D-Bus System Message Bus Socket.638gitlab # [ 4.103633] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.639gitlab # [ 4.105995] systemd[1]: Stopped Find NixOS closure.640gitlab # [ 4.107662] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio641gitlab # [ 4.111199] systemd[1]: Starting rw-sysroot-nix-store.service...642gitlab # [ 4.113246] systemd[1]: systemd-sysctl.service: Deactivated successfully.643gitlab # [ 4.115435] systemd[1]: Stopped Apply Kernel Variables.644gitlab # [ 4.117190] systemd[1]: systemd-modules-load.service: Deactivated successfully.645gitlab # [ 4.119135] systemd[1]: Stopped Load Kernel Modules.646gitlab # [ 4.121716] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.647gitlab # [ 4.123933] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.648gitlab # [ 4.127206] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.649gitlab # [ 4.129351] systemd[1]: Stopped Create System Files and Directories.650gitlab # [ 4.136613] systemd[1]: Stopped target Local File Systems.651gitlab # [ 4.138172] systemd[1]: Stopped target Preparation for Local File Systems.652gitlab # [ 4.139938] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.653gitlab # [ 4.141829] systemd[1]: Stopped Coldplug All udev Devices.654gitlab # [ 4.143953] systemd[1]: Stopping Rule-based Manager for Device Events and Files...655gitlab # [ 4.145943] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.656gitlab # [ 4.148092] systemd[1]: Stopped Virtual Console Setup.657gitlab # [ 4.154261] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.658gitlab # [ 4.157533] systemd[1]: Finished rw-sysroot-nix-store.service.659gitlab # [ 4.165525] systemd[1]: systemd-udevd.service: Deactivated successfully.660gitlab # [ 4.168258] systemd[1]: Stopped Rule-based Manager for Device Events and Files.661gitlab # [ 4.173580] systemd[1]: initrd-cleanup.service: Deactivated successfully.662gitlab # [ 4.176880] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.663gitlab # [ 4.185597] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.664gitlab # [ 4.186790] systemd[1]: Closed udev Control Socket.665gitlab # [ 4.192423] systemd[1]: Starting Cleanup udev Database...666gitlab # [ 4.193384] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.667gitlab # [ 4.194661] systemd[1]: Stopped Create Static Device Nodes in /dev.668gitlab # [ 4.197788] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.669gitlab # [ 4.199173] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.670gitlab # [ 4.204085] systemd[1]: kmod-static-nodes.service: Deactivated successfully.671gitlab # [ 4.205315] systemd[1]: Stopped Create List of Static Device Nodes.672gitlab # [ 4.216982] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.673gitlab # [ 4.219902] systemd[1]: Finished Cleanup udev Database.674gitlab # [ 4.224045] systemd[1]: Reached target Switch Root.675gitlab # [ 4.226990] systemd[1]: Starting NixOS Activation...676gitlab # [ 4.353243] initrd-nixos-activation-start[574]: booting system configuration /nix/store/8pw0dms91p54afz9cnbbxwzjm7p20wma-nixos-system-gitlab-test677gitlab # [ 4.396830] initrd-nixos-activation-start[574]: running activation script...678gitlab # [ 4.643349] initrd-nixos-activation-start[597]: setting up /etc...679gitlab # [ 4.726676] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.680gitlab # [ 4.729663] systemd[1]: Finished NixOS Activation.681gitlab # [ 4.739258] systemd[1]: Starting Switch Root...682gitlab # [ 4.771466] systemd[1]: Switching root.683gitlab # [ 5.168839] systemd-journald[150]: Received SIGTERM from PID 1 (systemd).684gitlab # [ 5.287886] NET: Registered PF_VSOCK protocol family685gitlab # [ 5.660771] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)686gitlab # [ 5.670889] systemd[1]: Detected virtualization kvm.687gitlab # [ 5.672720] systemd[1]: Detected architecture x86-64.688gitlab # [ 5.674718] systemd[1]: Detected first boot.689gitlab # [ 5.677308] systemd[1]: Initializing machine ID from random generator.690gitlab # [ 5.843176] systemd[1]: bpf-restrict-fs: LSM BPF program attached691gitlab # [ 5.911792] systemd[1]: Applying preset policy.692gitlab # [ 6.107577] systemd[1]: Populated /etc with preset unit settings.693gitlab # [ 6.210293] systemd[1]: initrd-switch-root.service: Deactivated successfully.694gitlab # [ 6.211725] systemd[1]: Stopped initrd-switch-root.service.695gitlab # [ 6.213971] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.696gitlab # [ 6.216124] systemd[1]: Created slice Slice /system/getty.697gitlab # [ 6.217585] systemd[1]: Created slice GitLab DevOps Platform Slice.698gitlab # [ 6.219121] systemd[1]: Created slice User and Session Slice.699gitlab # [ 6.220079] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.700gitlab # [ 6.221388] systemd[1]: Started Forward Password Requests to Wall Directory Watch.701gitlab # [ 6.222573] systemd[1]: Expecting device /dev/hvc0...702gitlab # [ 6.223386] systemd[1]: Expecting device /dev/ttyS0...703gitlab # [ 6.224271] systemd[1]: Reached target Local Encrypted Volumes.704gitlab # [ 6.225171] systemd[1]: Stopped target initrd-fs.target.705gitlab # [ 6.225981] systemd[1]: Stopped target initrd-root-fs.target.706gitlab # [ 6.226864] systemd[1]: Stopped target initrd-switch-root.target.707gitlab # [ 6.227864] systemd[1]: Reached target Virtual Machines and Containers.708gitlab # [ 6.228868] systemd[1]: Reached target Path Units.709gitlab # [ 6.229591] systemd[1]: Reached target Remote File Systems.710gitlab # [ 6.230456] systemd[1]: Reached target Slice Units.711gitlab # [ 6.231293] systemd[1]: Reached target Swaps.712gitlab # [ 6.232890] systemd[1]: Listening on Process Core Dump Socket.713gitlab # [ 6.234279] systemd[1]: Listening on Credential Encryption/Decryption.714gitlab # [ 6.236824] systemd[1]: Starting Journal Log Access Socket...715gitlab # [ 6.238226] systemd[1]: Listening on Journal Audit Socket.716gitlab # [ 6.239278] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.717gitlab # [ 6.240474] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki718gitlab # [ 6.242093] systemd[1]: Listening on udev Control Socket.719gitlab # [ 6.246727] systemd[1]: Mounting Huge Pages File System...720gitlab # [ 6.251369] systemd[1]: Mounting POSIX Message Queue File System...721gitlab # [ 6.255976] systemd[1]: Mounting Kernel Debug File System...722gitlab # [ 6.268628] systemd[1]: Mounting Kernel Trace File System...723gitlab # [ 6.273826] systemd[1]: Starting Create List of Static Device Nodes...724gitlab # [ 6.276696] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio725gitlab # [ 6.284103] systemd[1]: Starting Load Kernel Module configfs...726gitlab # [ 6.286608] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm727gitlab # [ 6.290212] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore728gitlab # [ 6.294106] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse729gitlab # [ 6.300011] systemd[1]: Mounting FUSE Control File System...730gitlab # [ 6.302283] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67731gitlab # [ 6.311308] systemd[1]: Starting Journal Service...732gitlab # [ 6.315002] systemd[1]: Starting Load Kernel Modules...733gitlab # [ 6.319325] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...734gitlab # [ 6.323939] systemd[1]: Starting Remount Root and Kernel File Systems...735gitlab # [ 6.327027] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki736gitlab # [ 6.334281] systemd[1]: Starting Coldplug All udev Devices...737gitlab # [ 6.336179] systemd-journald[793]: Collecting audit messages is enabled.738gitlab # [ 6.345214] systemd[1]: Listening on Journal Log Access Socket.739gitlab # [ 6.213188] systemd[1]: Queued start job for default target Multi-User System.740gitlab # [ 6.215820] systemd[1]: Unnecessary job was removed for /dev/vdb.741gitlab # [ 6.218716] systemd[1]: systemd-journald.service: Deactivated successfully.742gitlab # [ 6.221903] systemd-modules-load[794]: Using 3 probe threads743gitlab # [ 6.224157] systemd-modules-load[794]: Inserted module 'tls'744gitlab # [ 6.345632] loop: module loaded745gitlab # [ 6.346313] systemd[1]: Mounted Huge Pages File System.746gitlab # [ 6.377091] systemd[1]: Started Journal Service.747gitlab # [ 6.239582] systemd-modules-load[794]: Inserted module 'loop'748gitlab # [ 6.243121] systemd[1]: Mounted POSIX Message Queue File System.749gitlab # [ 6.384102] EXT4-fs (vda): re-mounted bab2b030-ad85-4218-8e00-0d421fc6cb71.750gitlab # [ 6.246208] systemd[1]: Mounted Kernel Debug File System.751gitlab # [ 6.249286] systemd[1]: Mounted Kernel Trace File System.752gitlab # [ 6.251214] systemd[1]: Finished Create List of Static Device Nodes.753gitlab # [ 6.253151] systemd[1]: modprobe@configfs.service: Deactivated successfully.754gitlab # [ 6.255203] systemd[1]: Finished Load Kernel Module configfs.755gitlab # [ 6.256936] systemd[1]: Mounted FUSE Control File System.756gitlab # [ 6.258584] systemd[1]: Finished Load Kernel Modules.757gitlab # [ 6.260166] systemd[1]: Finished Remount Root and Kernel File Systems.758gitlab # [ 6.262079] systemd[1]: Mounting Kernel Configuration File System...759gitlab # [ 6.263952] systemd[1]: Starting Firewall...760gitlab # [ 6.266511] systemd[1]: Starting Flush Journal to Persistent Storage...761gitlab # [ 6.270190] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore762gitlab # [ 6.276145] systemd[1]: Starting Load/Save OS Random Seed...763gitlab # [ 6.277427] systemd[1]: Starting Apply Kernel Variables...764gitlab # [ 6.278652] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...765gitlab # [ 6.279808] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki766gitlab # [ 6.281135] systemd[1]: Mounted Kernel Configuration File System.767gitlab # [ 6.282563] systemd-oomd[795]: No swap; memory pressure usage will be degraded768gitlab # [ 6.284293] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.769gitlab # [ 6.287713] systemd[1]: Finished Load/Save OS Random Seed.770gitlab # [ 6.289322] systemd[1]: Reached target First Boot Complete.771gitlab # [ 6.429967] systemd-journald[793]: Received client request to flush runtime journal.772gitlab # [ 6.329028] systemd[1]: Finished Apply Kernel Variables.773gitlab # [ 6.330743] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.774gitlab # [ 6.331767] systemd[1]: Starting Create Static Device Nodes in /dev...775gitlab # [ 6.332660] systemd[1]: Finished Flush Journal to Persistent Storage.776gitlab # [ 6.339834] systemd[1]: Finished Create Static Device Nodes in /dev.777gitlab # [ 6.340795] systemd[1]: Reached target Preparation for Local File Systems.778gitlab # [ 6.342834] systemd[1]: Starting Rule-based Manager for Device Events and Files...779gitlab # [ 6.375965] systemd-udevd[844]: Using default interface naming scheme 'v260'.780gitlab # [ 6.401622] systemd[1]: Finished Coldplug All udev Devices.781gitlab # [ 6.408434] systemd[1]: Started Rule-based Manager for Device Events and Files.782gitlab # [ 6.488612] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse783gitlab # [ 6.504636] (udev-worker)[922]: Network interface NamePolicy= disabled on kernel command line.784gitlab # [ 6.506240] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.785gitlab # [ 6.513068] systemd[1]: Finished Firewall.786gitlab # [ 6.516069] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.787gitlab # [ 6.520465] (udev-worker)[911]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.788gitlab # [ 6.523066] (udev-worker)[911]: Network interface NamePolicy= disabled on kernel command line.789gitlab # [ 6.536261] systemd[1]: Condition check resulted in Virtio network device being skipped.790gitlab # [ 6.537696] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore791gitlab # [ 6.539323] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67792gitlab # [ 6.541606] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore793gitlab # [ 6.543205] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki794gitlab # [ 6.544475] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki795gitlab # [ 6.704218] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3796gitlab # [ 6.716241] Floppy drive(s): fd0 is 2.88M AMI BIOS797gitlab # [ 6.727012] ACPI: button: Power Button [PWRF]798gitlab # [ 6.727033] rtc_cmos 00:05: RTC can wake from S4799gitlab # [ 6.729169] parport_pc 00:03: reported by Plug and Play ACPI800gitlab # [ 6.729334] FDC 0 is a S82078B801gitlab # [ 6.731268] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]802gitlab # [ 6.735566] rtc_cmos 00:05: registered as rtc0803gitlab # [ 6.736435] rtc_cmos 00:05: setting system clock to 2026-07-21T06:33:47 UTC (1784615627)804gitlab # [ 6.736560] mousedev: PS/2 mouse device common for all mice805gitlab # [ 6.741576] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs806gitlab # [ 6.758171] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0807gitlab # [ 6.759467] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD808gitlab # [ 6.763041] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5809gitlab # [ 6.765486] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4810gitlab # [ 6.771106] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console811gitlab # [ 6.773638] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6812gitlab # [ 6.801233] Console: switching to colour dummy device 80x25813gitlab # [ 6.819877] [drm] Found bochs VGA, ID 0xb0c5.814gitlab # [ 6.819879] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.815gitlab # [ 6.822117] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic816gitlab # [ 6.822861] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0817gitlab # [ 6.685823] systemd[1]: Starting Virtual Console Setup...818gitlab # [ 6.839741] fbcon: bochs-drmdrmfb (fb0) is primary device819gitlab # [ 6.869550] ppdev: user-space parallel port driver820gitlab # [ 6.873930] Console: switching to colour frame buffer device 160x50821gitlab # [ 6.876404] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device822gitlab # [ 6.740719] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.823gitlab # [ 6.743993] systemd[1]: Stopped Virtual Console Setup.824gitlab # [ 6.749226] systemd[1]: Starting Virtual Console Setup...825gitlab # [ 6.904062] kvm_amd: TSC scaling supported826gitlab # [ 6.904736] kvm_amd: Nested Virtualization enabled827gitlab # [ 6.905365] kvm_amd: Nested Paging enabled828gitlab # [ 6.905981] kvm_amd: LBR virtualization supported829gitlab # [ 6.906685] kvm_amd: Virtual VMLOAD VMSAVE supported830gitlab # [ 6.907359] kvm_amd: Virtual GIF supported831gitlab # [ 6.908030] kvm_amd: Virtual NMI enabled832gitlab # [ 6.941736] EDAC MC: Ver: 3.0.0833gitlab # [ 6.811241] systemd-vconsole-setup[966]: Configuration of first virtual console was skipped, ignoring remaining ones.834gitlab # [ 6.814177] systemd[1]: Finished Virtual Console Setup.835gitlab # [ 7.077225] systemd[1]: Mounting /run/wrappers...836gitlab # [ 7.102432] systemd[1]: Mounted /run/wrappers.837gitlab # [ 7.104470] systemd[1]: Reached target Local File Systems.838gitlab # [ 7.106343] systemd[1]: Listening on Boot Loader Control Service Socket.839gitlab # [ 7.108892] systemd[1]: Starting register-nix-paths.service...840gitlab # [ 7.111385] systemd[1]: Starting Create SUID/SGID Wrappers...841gitlab # [ 7.113369] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.842gitlab # [ 7.116281] systemd[1]: Starting Save Transient machine-id to Disk...843gitlab # [ 7.120320] systemd[1]: Starting Create System Files and Directories...844gitlab # [ 7.167290] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.845gitlab # [ 7.171179] systemd[1]: Finished Save Transient machine-id to Disk.846gitlab # [ 7.175576] systemd[1]: Finished Create System Files and Directories.847gitlab # [ 7.181140] systemd[1]: Starting Rebuild Journal Catalog...848gitlab # [ 7.183084] systemd[1]: Starting Record System Boot/Shutdown in UTMP...849gitlab # [ 7.208443] systemd[1]: Finished Record System Boot/Shutdown in UTMP.850gitlab # [ 7.227428] systemd[1]: Finished Rebuild Journal Catalog.851gitlab # [ 7.229367] systemd[1]: Starting Update is Completed...852gitlab # [ 7.245859] systemd[1]: Finished Update is Completed.853gitlab # [ 7.284034] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.854gitlab # [ 7.285392] systemd[1]: Finished Create SUID/SGID Wrappers.855gitlab # [ 7.286221] systemd[1]: suid-sgid-wrappers.service: Consumed 135ms CPU time over 172ms wall clock time, 4.6M memory peak, 1.1M read from disk.856gitlab # [ 7.522092] systemd[1]: Finished register-nix-paths.service.857gitlab # [ 7.524426] systemd[1]: Reached target System Initialization.858gitlab # [ 7.526498] systemd[1]: Started Discard unused filesystem blocks once a week.859gitlab # [ 7.528902] systemd[1]: Started logrotate.timer.860gitlab # [ 7.530576] systemd[1]: Started Daily Cleanup of Temporary Directories.861gitlab # [ 7.532783] systemd[1]: Reached target Timer Units.862gitlab # [ 7.534509] systemd[1]: Listening on D-Bus System Message Bus Socket.863gitlab # [ 7.536608] systemd[1]: Listening on Nix Daemon Socket.864gitlab # [ 7.541662] systemd[1]: Starting nixbot CI service socket...865gitlab # [ 7.543646] systemd[1]: Listening on Hostname Service Socket.866gitlab # [ 7.545801] systemd[1]: Starting D-Bus System Message Bus...867gitlab # [ 7.547795] systemd[1]: Listening on nixbot CI service socket.868gitlab # [ 7.549818] systemd[1]: Reached target Socket Units.869gitlab # [ 7.587661] dbus-broker-launch[1056]: Looking up NSS user entry for 'systemd-timesync'...870gitlab # [ 7.590608] dbus-broker-launch[1056]: NSS returned no entry for 'systemd-timesync'871gitlab # [ 7.593386] dbus-broker-launch[1056]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"872gitlab # [ 7.598514] systemd[1]: Started D-Bus System Message Bus.873gitlab # [ 7.613930] dbus-broker-launch[1056]: Ready874gitlab # [ 7.617669] systemd[1]: Reached target Basic System.875gitlab # [ 7.620727] systemd[1]: Started backdoor.service.876gitlab # [ 7.623718] systemd[1]: Starting gitlab-config.service...877gitlab # [ 7.626248] systemd[1]: Starting Import lastlog data into lastlog2 database...878gitlab # [ 7.632348] systemd[1]: Starting Logrotate configuration check...879gitlab # [ 7.646351] systemd[1]: Starting Name Service Cache Daemon (nsncd)...880gitlab # [ 7.650975] systemd[1]: Starting Post-Boot Actions...881gitlab # [ 7.653791] logrotate[1061]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.882gitlab # [ 7.658558] logrotate[1061]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf883gitlab # [ 7.661374] logrotate[1061]: note: 'monthly' overrides previously specified 'weekly'884gitlab # [ 7.663617] logrotate[1061]: Reading state from file: /var/lib/logrotate.status885gitlab # [ 7.666108] logrotate[1061]: state file /var/lib/logrotate.status does not exist886gitlab # [ 7.667971] logrotate[1061]: Allocating hash table for state file, size 64 entries887gitlab # connecting to host...888gitlab # [ 7.669801] logrotate[1061]: Handling 4 logs889gitlab # [ 7.670386] logrotate[1061]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed890gitlab # [ 7.671658] logrotate[1061]: considering log /var/log/btmp891gitlab # [ 7.673811] logrotate[1061]: Creating new state892gitlab # [ 7.674993] logrotate[1061]: Now: 2026-07-21 06:33893gitlab # [ 7.675872] logrotate[1061]: Last rotated at 2026-07-21 06:00894gitlab # [ 7.680120] logrotate[1061]: log does not need rotating (log has already been rotated)895gitlab # [ 7.682285] logrotate[1061]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed896gitlab # [ 7.685476] logrotate[1061]: considering log /var/log/wtmp897gitlab # [ 7.686354] logrotate[1061]: Creating new state898gitlab # [ 7.687051] logrotate[1061]: Now: 2026-07-21 06:33899gitlab # [ 7.687805] logrotate[1061]: Last rotated at 2026-07-21 06:00900gitlab: Guest shell says: b'Spawning backdoor root shell...\n'901gitlab: connected to guest root shell902gitlab # [ 7.688662] logrotate[1061]: log does not need rotating (log has already been rotated)903gitlab: (connecting took 8.32 seconds)904gitlab # [ 7.689868] logrotate[1061]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed905gitlab: (finished: waiting for the VM to finish booting, in 8.32 seconds)906gitlab # [ 7.691619] logrotate[1061]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1061)907gitlab # [ 7.692709] logrotate[1061]: considering log /var/gitlab/state/log/*.log908gitlab # [ 7.693582] logrotate[1061]: log /var/gitlab/state/log/*.log does not exist -- skipping909gitlab # [ 7.694560] logrotate[1061]: Creating new state910gitlab # [ 7.695218] logrotate[1061]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1061)911gitlab # [ 7.696323] logrotate[1061]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed912gitlab # [ 7.697953] logrotate[1061]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1061)913gitlab # [ 7.699059] logrotate[1061]: considering log /var/log/nginx/*.log914gitlab # [ 7.699842] logrotate[1061]: log /var/log/nginx/*.log does not exist -- skipping915gitlab # [ 7.701037] logrotate[1061]: Creating new state916gitlab # [ 7.701841] logrotate[1061]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1061)917gitlab # [ 7.703167] systemd[1]: Started Reset console on configuration changes.918gitlab # [ 7.704237] systemd[1]: Starting resolvconf update...919gitlab # [ 7.705388] systemd[1]: Finished Logrotate configuration check.920gitlab # [ 7.706468] systemd[1]: Finished Post-Boot Actions.921gitlab # [ 7.707347] systemd[1]: Finished Import lastlog data into lastlog2 database.922gitlab # [ 7.710779] nsncd[1062]: Jul 21 06:33:48.612 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"923gitlab # [ 7.712996] systemd[1]: Started Name Service Cache Daemon (nsncd).924gitlab # [ 7.714039] systemd[1]: Reached target Host and Network Name Lookups.925gitlab # [ 7.715080] systemd[1]: Reached target User and Group Name Lookups.926gitlab # [ 7.716128] systemd[1]: Starting User Login Management...927gitlab # [ 7.745736] systemd[1]: Stopped target Host and Network Name Lookups.928gitlab # [ 7.746792] systemd[1]: Stopping Host and Network Name Lookups...929gitlab # [ 7.747853] systemd[1]: Stopped target User and Group Name Lookups.930gitlab # [ 7.748769] systemd[1]: Stopping User and Group Name Lookups...931gitlab # [ 7.750199] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...932gitlab # [ 7.750822] systemd[1]: nscd.service: Deactivated successfully.933gitlab # [ 7.752831] systemd[1]: Stopped Name Service Cache Daemon (nsncd).934gitlab # [ 7.753872] systemd[1]: nscd.service: Consumed 20ms CPU time over 103ms wall clock time, 7.9M memory peak, 5.6M read from disk.935gitlab # [ 7.758641] systemd[1]: Starting Name Service Cache Daemon (nsncd)...936gitlab # [ 7.761136] systemd-logind[1105]: New seat seat0.937gitlab # [ 7.765449] systemd-logind[1105]: Watching system buttons on /dev/input/event2 (Power Button)938gitlab # [ 7.766743] systemd-logind[1105]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)939gitlab # [ 7.767982] systemd-logind[1105]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)940gitlab # [ 7.769378] systemd[1]: Started User Login Management.941gitlab # [ 7.771605] systemd[1]: Starting linger-users.service...942gitlab # [ 7.780791] nsncd[1150]: Jul 21 06:33:48.682 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"943gitlab # [ 7.781231] systemd[1]: Finished resolvconf update.944gitlab # [ 7.781813] systemd[1]: Started Name Service Cache Daemon (nsncd).945gitlab # [ 7.782447] systemd[1]: Reached target Preparation for Network.946gitlab # [ 7.783035] systemd[1]: Reached target Host and Network Name Lookups.947gitlab # [ 7.783524] systemd[1]: Reached target User and Group Name Lookups.948gitlab # [ 7.784463] systemd[1]: Starting DHCP Client...949gitlab # [ 7.786618] systemd[1]: Starting Address configuration of eth1...950gitlab # [ 7.788311] systemd[1]: Starting Extra networking commands....951gitlab # [ 7.793719] systemd[1]: linger-users.service: Deactivated successfully.952gitlab # [ 7.794169] systemd[1]: Finished linger-users.service.953gitlab # [ 7.812362] network-addresses-eth1-start[1173]: adding address 192.168.1.1/24... done954gitlab # [ 7.816904] network-addresses-eth1-start[1173]: adding address 2001:db8:1::1/64... done955gitlab # [ 7.822625] systemd[1]: Finished Address configuration of eth1.956gitlab # [ 7.846720] systemd[1]: Finished Extra networking commands..957gitlab # [ 7.848343] systemd[1]: Reached target Network.958gitlab # [ 7.851347] systemd[1]: Starting gitlab-workhorse.service...959gitlab # [ 7.853337] systemd[1]: Starting Nginx Web Server...960gitlab # [ 7.854523] dhcpcd[1183]: dhcpcd-10.3.2 starting961gitlab # [ 7.857773] dhcpcd[1234]: dev: loaded udev962gitlab # [ 7.859323] systemd[1]: Starting PostgreSQL Server...963gitlab # [ 7.861852] systemd[1]: Starting Redis Server - redis-gitlab...964gitlab # [ 7.862276] systemd[1]: Starting Permit User Sessions...965gitlab # [ 8.006237] 8021q: 802.1Q VLAN Support v1.8966gitlab # [ 8.006884] 8021q: adding VLAN 0 to HW filter on device eth1967gitlab # [ 7.881222] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.968gitlab # [ 7.883860] systemd[1]: Finished Permit User Sessions.969gitlab # [ 7.888541] systemd[1]: Started Getty on tty1.970gitlab # [ 7.889675] systemd[1]: Reached target Login Prompts.971gitlab # [ 8.096025] cfg80211: Loading compiled-in X.509 certificates for regulatory database972gitlab # [ 8.101896] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'973gitlab # [ 8.102918] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'974gitlab # [ 8.104966] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2975gitlab # [ 8.106069] cfg80211: failed to load regulatory.db976gitlab # [ 7.983078] redis-server[1255]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.977gitlab # [ 7.983389] redis-server[1255]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo978gitlab # [ 7.984232] redis-server[1255]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1255, just started979gitlab # [ 7.984800] redis-server[1255]: Configuration loaded980gitlab # [ 7.985408] redis[1255]: monotonic clock: POSIX clock_gettime981gitlab # [ 7.985928] redis[1255]: Running mode=standalone, port=0.982gitlab # [ 7.989262] nginx-pre-start[1258]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok983gitlab # [ 7.989929] nginx-pre-start[1258]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful984gitlab # [ 7.990307] systemd[1]: Started Redis Server - redis-gitlab.985gitlab # [ 7.990813] redis[1255]: Server initialized986gitlab # [ 7.991448] redis[1255]: Ready to accept connections unix987gitlab # [ 7.991986] redis[1255]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.988gitlab # [ 7.997297] systemd[1]: Started Nginx Web Server.989gitlab # [ 8.143741] 8021q: adding VLAN 0 to HW filter on device eth0990gitlab # [ 8.006891] dhcpcd[1234]: eth0: waiting for carrier991gitlab # [ 8.007308] dhcpcd[1234]: eth0: carrier acquired992gitlab # [ 8.009568] dhcpcd[1234]: DUID 00:01:00:01:31:f1:cf:4c:52:54:00:12:34:56993gitlab # [ 8.009974] dhcpcd[1234]: eth0: IAID 00:12:34:56994gitlab # [ 8.010714] dhcpcd[1234]: eth0: adding address fe80::5054:ff:fe12:3456995gitlab # [ 8.015889] postgresql-pre-start[1259]: The files belonging to this database system will be owned by user "postgres".996gitlab # [ 8.016276] postgresql-pre-start[1259]: This user must also own the server process.997gitlab # [ 8.019513] postgresql-pre-start[1259]: The database cluster will be initialized with locale "en_US.UTF-8".998gitlab # [ 8.019931] postgresql-pre-start[1259]: The default database encoding has accordingly been set to "UTF8".999gitlab # [ 8.020544] postgresql-pre-start[1259]: The default text search configuration will be set to "english".1000gitlab # [ 8.021171] postgresql-pre-start[1259]: Data page checksums are disabled.1001gitlab # [ 8.023931] postgresql-pre-start[1259]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1002gitlab # [ 8.026118] postgresql-pre-start[1259]: creating subdirectories ... ok1003gitlab # [ 8.027233] postgresql-pre-start[1259]: selecting dynamic shared memory implementation ... posix1004gitlab # [ 8.051942] postgresql-pre-start[1259]: selecting default "max_connections" ... 1001005gitlab # [ 8.094401] postgresql-pre-start[1259]: selecting default "shared_buffers" ... 128MB1006gitlab # [ 8.344522] systemd[1]: Started gitlab-workhorse.service.1007gitlab # [ 8.451452] postgresql-pre-start[1259]: selecting default time zone ... UTC1008gitlab # [ 8.453992] postgresql-pre-start[1259]: creating configuration files ... ok1009gitlab # [ 8.465597] dhcpcd[1234]: eth0: soliciting a DHCP lease1010gitlab # [ 8.610254] NET: Registered PF_PACKET protocol family1011gitlab # [ 8.475752] dhcpcd[1234]: eth0: offered 10.0.2.15 from 10.0.2.21012gitlab # [ 8.482237] dhcpcd[1234]: eth0: probing address 10.0.2.15/241013gitlab # [ 8.572974] gitlab-workhorse[1292]: time="2026-07-21T06:33:49Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41014gitlab # [ 8.576587] gitlab-workhorse[1292]: time="2026-07-21T06:33:49Z" level=info msg="Using redis/go-redis"1015gitlab # [ 8.579567] gitlab-workhorse[1292]: time="2026-07-21T06:33:49Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1016gitlab # [ 8.638889] postgresql-pre-start[1259]: running bootstrap script ... ok1017gitlab # [ 9.043767] postgresql-pre-start[1259]: performing post-bootstrap initialization ... ok1018gitlab # [ 9.355522] postgresql-pre-start[1259]: syncing data to disk ... ok1019gitlab # [ 9.357890] postgresql-pre-start[1259]: initdb: warning: enabling "trust" authentication for local connections1020gitlab # [ 9.361073] postgresql-pre-start[1259]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1021gitlab # [ 9.365948] postgresql-pre-start[1259]: Success. You can now start the database server using:1022gitlab # [ 9.368701] postgresql-pre-start[1259]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1023gitlab # [ 9.453345] postgres[1317]: [1317] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1024gitlab # [ 9.457609] postgres[1317]: [1317] LOG: listening on IPv6 address "::1", port 54321025gitlab # [ 9.460373] postgres[1317]: [1317] LOG: listening on IPv4 address "127.0.0.1", port 54321026gitlab # [ 9.463138] postgres[1317]: [1317] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1027gitlab # [ 9.468319] postgres[1329]: [1329] LOG: database system was shut down at 2026-07-21 06:33:49 GMT1028gitlab # [ 9.481159] postgres[1317]: [1317] LOG: database system is ready to accept connections1029gitlab # [ 9.486463] systemd[1]: Started PostgreSQL Server.1030gitlab # [ 9.490446] systemd[1]: Starting PostgreSQL Setup Scripts...1031gitlab # [ 9.560439] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1322]: mkdir -p /var/gitlab/state/home/.ssh: OK1032gitlab # [ 9.566422] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1322]: chmod 700 /var/gitlab/state/home/.ssh: OK1033gitlab # [ 9.629146] systemd[1]: Finished gitlab-config.service.1034gitlab # [ 9.632446] systemd[1]: Started gitaly.service.1035gitlab # [ 9.662222] postgresql-setup-start[1344]: CREATE DATABASE1036gitlab # [ 9.690680] postgresql-setup-start[1359]: CREATE ROLE1037gitlab # [ 9.705557] postgresql-setup-start[1361]: ALTER DATABASE1038gitlab # [ 9.738103] postgresql-setup-start[1366]: CREATE ROLE1039gitlab # [ 9.741109] systemd[1]: Finished PostgreSQL Setup Scripts.1040gitlab # [ 9.742896] systemd[1]: Reached target PostgreSQL.1041gitlab # [ 9.746034] systemd[1]: Starting gitlab-db-config.service...1042gitlab # [ 9.749654] systemd[1]: Starting gitlab-postgresql.service...1043gitlab # [ 9.861103] gitlab-postgresql-start[1376]: CREATE DATABASE1044gitlab # [ 9.893089] gitlab-postgresql-start[1395]: CREATE EXTENSION1045gitlab # [ 9.933360] gitlab-postgresql-start[1397]: CREATE EXTENSION1046gitlab # [ 9.935432] systemd[1]: Finished gitlab-postgresql.service.1047gitlab # [ 9.936612] dhcpcd[1234]: eth0: soliciting an IPv6 router1048gitlab # [ 9.937503] dhcpcd[1234]: eth0: Router Advertisement from fe80::21049gitlab # [ 9.938430] dhcpcd[1234]: eth0: adding address fec0::5054:ff:fe12:3456/641050gitlab # [ 9.939380] dhcpcd[1234]: eth0: adding route to fec0::/641051gitlab # [ 9.940283] dhcpcd[1234]: eth0: adding default route via fe80::21052gitlab # [ 10.182172] gitaly[1354]: time="2026-07-21T06:33:51.083Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13541053gitlab # [ 10.185860] gitaly[1354]: time="2026-07-21T06:33:51.084Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13541054gitlab # [ 10.187424] gitaly[1354]: time="2026-07-21T06:33:51.084Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13541055gitlab # [ 10.189498] gitaly[1354]: time="2026-07-21T06:33:51.084Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13541056gitlab # [ 10.191245] gitaly[1354]: time="2026-07-21T06:33:51.084Z" level=info msg="Starting Gitaly" pid=1354 version=18.11.41057gitlab # [ 10.370174] gitaly[1354]: time="2026-07-21T06:33:51.272Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=186 pid=13541058gitlab # [ 10.372162] gitaly[1354]: time="2026-07-21T06:33:51.273Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13541059gitlab # [ 10.373917] gitaly[1354]: time="2026-07-21T06:33:51.273Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13541060gitlab # [ 10.375623] gitaly[1354]: time="2026-07-21T06:33:51.273Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1354/git-exec-3329554451.d/git pid=13541061gitlab # [ 10.377866] gitaly[1354]: time="2026-07-21T06:33:51.277Z" level=info msg="finished detecting git version" duration_ms=4 pid=13541062gitlab # [ 10.379630] gitaly[1354]: time="2026-07-21T06:33:51.277Z" level=info msg="using Git version" pid=1354 version=2.53.g1063gitlab # [ 10.381243] gitaly[1354]: time="2026-07-21T06:33:51.278Z" level=info msg="clearing disk cache object folder" pid=1354 storage=default1064gitlab # [ 10.383117] gitaly[1354]: time="2026-07-21T06:33:51.278Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1242462252 pid=1354 storage=default1065gitlab # [ 10.385931] gitaly[1354]: time="2026-07-21T06:33:51.278Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1242462252 pid=1354 storage=default1066gitlab # [ 10.389085] gitaly[1354]: time="2026-07-21T06:33:51.278Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13541067gitlab # [ 10.391920] gitaly[1354]: time="2026-07-21T06:33:51.278Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13541068gitlab # [ 10.394788] gitaly[1354]: time="2026-07-21T06:33:51.278Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1242462252 clear_duration_ms=0 pid=1354 storage=default1069gitlab # [ 10.421397] gitaly[1354]: time="2026-07-21T06:33:51.323Z" level=info msg="counting repositories" pid=13541070gitlab # [ 10.423181] gitaly[1354]: time="2026-07-21T06:33:51.323Z" level=info msg="finished tempdir cleaner walk" pid=1354 storage=default time_ms=01071gitlab # [ 10.423547] gitaly[1354]: time="2026-07-21T06:33:51.323Z" level=info msg="starting to count repositories" pid=1354 storage_path=/var/gitlab/state/repositories1072gitlab # [ 10.424259] gitaly[1354]: time="2026-07-21T06:33:51.323Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1354 storage_path=/var/gitlab/state/repositories1073gitlab # [ 10.425152] gitaly[1354]: time="2026-07-21T06:33:51.323Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13541074gitlab # [ 10.425681] gitaly[1354]: time="2026-07-21T06:33:51.323Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13541075gitlab # [ 10.426271] gitaly[1354]: time="2026-07-21T06:33:51.323Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1354 schema=unix1076gitlab # [ 10.426583] gitaly[1354]: time="2026-07-21T06:33:51.324Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1354/sock.d/intern pid=1354 schema=unix1077gitlab # [ 10.427255] gitaly[1354]: time="2026-07-21T06:33:51.325Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13541078gitlab # [ 10.427961] gitaly[1354]: time="2026-07-21T06:33:51.325Z" level=info msg="Started Gitaly" duration_ms=241 pid=13541079gitlab # [ 10.428804] gitaly[1354]: time="2026-07-21T06:33:51.325Z" level=info msg="maintenance: daily scheduled" pid=1354 scheduled="2026-07-21 12:00:00 +0000 UTC"1080gitlab # [ 11.281789] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1371]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1081gitlab # [ 11.282362] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1371]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1082gitlab # [ 11.327485] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1371]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1083gitlab # [ 12.283461] gitaly[1354]: time="2026-07-21T06:33:53.184Z" level=info msg="License database preloaded" duration_ms=2100 pid=13541084gitlab # [ 14.083147] dhcpcd[1234]: eth0: leased 10.0.2.15 for 86400 seconds1085gitlab # [ 14.083562] dhcpcd[1234]: eth0: adding route to 10.0.2.0/241086gitlab # [ 14.085276] dhcpcd[1234]: eth0: adding default route via 10.0.2.21087gitlab # [ 14.189568] systemd[1]: Started DHCP Client.1088gitlab # [ 14.192049] systemd[1]: Reached target Network is Online.1089gitlab # [ 40.543630] postgres[1740]: [1740] ERROR: relation "feature_gates" does not exist at character 5231090gitlab # [ 40.544265] postgres[1740]: [1740] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1091gitlab # [ 40.545617] postgres[1740]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1092gitlab # [ 40.548076] postgres[1740]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1093gitlab # [ 40.549983] postgres[1740]: attidentity AS identity,1094gitlab # [ 40.551897] postgres[1740]: attgenerated as attgenerated1095gitlab # [ 40.553441] postgres[1740]: FROM pg_attribute a1096gitlab # [ 40.555349] postgres[1740]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1097gitlab # [ 40.556927] postgres[1740]: LEFT JOIN pg_type t ON a.atttypid = t.oid1098gitlab # [ 40.559218] postgres[1740]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1099gitlab # [ 40.561151] postgres[1740]: WHERE a.attrelid = '"feature_gates"'::regclass1100gitlab # [ 40.563090] postgres[1740]: AND a.attnum > 0 AND NOT a.attisdropped1101gitlab # [ 40.564938] postgres[1740]: ORDER BY a.attnum1102gitlab # [ 40.567330] postgres[1740]:1103gitlab # [ 40.741782] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1371]: W, [2026-07-21T06:34:21.642953 #1371] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1104gitlab # [ 45.548485] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1786]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1105gitlab # [ 45.552326] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1786]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1106gitlab # [ 48.631876] gitlab-workhorse[1292]: time="2026-07-21T06:34:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1107gitlab # [ 58.641427] gitlab-workhorse[1292]: time="2026-07-21T06:34:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1108gitlab # [ 59.412515] postgres[1852]: [1852] ERROR: relation "table_without_models" does not exist1109gitlab # [ 59.413234] postgres[1852]: [1852] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1110gitlab # [ 68.651599] gitlab-workhorse[1292]: time="2026-07-21T06:34:49Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1111gitlab # [ 73.126131] systemd[1]: Finished gitlab-db-config.service.1112gitlab # [ 73.131516] systemd[1]: Started gitlab-sidekiq.service.1113gitlab # [ 73.136402] systemd[1]: Starting gitlab.service...1114gitlab # [ 73.461281] bundle[2037]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1115gitlab # [ 73.462295] bundle[2037]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1116gitlab # [ 73.466100] bundle[2037]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1117gitlab # [ 76.159306] sidekiq-cluster[2036]: {"severity":"INFO","time":"2026-07-21T06:34:57.044Z","message":"Starting cluster with 1 processes"}1118gitlab # [ 76.235267] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.117Z","pid":2037,"message":"* Enabling systemd notification integration"}1119gitlab # [ 76.236100] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"Puma starting in cluster mode..."}1120gitlab # [ 76.236945] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1121gitlab # [ 76.237785] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1122gitlab # [ 76.238742] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"* Min threads: 0"}1123gitlab # [ 76.239676] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"* Max threads: 4"}1124gitlab # [ 76.240523] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"* Environment: production"}1125gitlab # [ 76.241216] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"* Master PID: 2037"}1126gitlab # [ 76.241884] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"* Workers: 2"}1127gitlab # [ 76.242772] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1128gitlab # [ 76.243463] bundle[2037]: {"timestamp":"2026-07-21T06:34:57.136Z","pid":2037,"message":"* Preloading application"}1129gitlab # [ 78.660823] gitlab-workhorse[1292]: time="2026-07-21T06:34:59Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1130gitlab # [ 88.671277] gitlab-workhorse[1292]: time="2026-07-21T06:35:09Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1131gitlab # [ 98.327970] bundle[2037]: W, [2026-07-21T06:35:19.229125 #2037] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1132gitlab # [ 98.672648] gitlab-workhorse[1292]: time="2026-07-21T06:35:19Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1133gitlab # [ 101.134249] sidekiq-cluster[2058]: 2026-07-21T06:35:22.035Z pid=2058 tid=txe INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2058, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>429621248}1134gitlab # [ 101.312214] sidekiq-cluster[2058]: W, [2026-07-21T06:35:22.213975 #2058] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1135gitlab # [ 101.353859] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:22.255Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1136gitlab # [ 103.165440] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.066Z","message":"GitLab reliable fetch activated!"}1137gitlab # [ 103.165855] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.066Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1138gitlab # [ 103.197152] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.099Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1139gitlab # [ 103.204609] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.106Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1140gitlab # [ 103.214049] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.115Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1141gitlab # [ 103.218289] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.120Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1142gitlab # [ 103.223646] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.125Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1143gitlab # [ 103.228027] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.129Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1144gitlab # [ 103.231725] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.133Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1145gitlab # [ 103.235690] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.137Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1146gitlab # [ 103.239429] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.141Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1147gitlab # [ 103.242400] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.144Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1148gitlab # [ 103.245648] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.147Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1149gitlab # [ 103.250916] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.152Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1150gitlab # [ 103.255279] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.157Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1151gitlab # [ 103.259253] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.161Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1152gitlab # [ 103.263039] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.164Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1153gitlab # [ 103.266764] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.168Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1154gitlab # [ 103.269977] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.171Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1155gitlab # [ 103.274766] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.176Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1156gitlab # [ 103.278220] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.180Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1157gitlab # [ 103.281542] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.183Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1158gitlab # [ 103.284806] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.186Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1159gitlab # [ 103.288121] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.190Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1160gitlab # [ 103.290422] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.192Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1161gitlab # [ 103.292418] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.194Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1162gitlab # [ 103.294253] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.196Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1163gitlab # [ 103.296306] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.198Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1164gitlab # [ 103.298768] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.200Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1165gitlab # [ 103.300729] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.202Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1166gitlab # [ 103.302563] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.204Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1167gitlab # [ 103.304730] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.206Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1168gitlab # [ 103.306554] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.208Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1169gitlab # [ 103.308464] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.210Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1170gitlab # [ 103.310912] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.212Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1171gitlab # [ 103.312831] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.214Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1172gitlab # [ 103.315169] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.217Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1173gitlab # [ 103.316762] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.218Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1174gitlab # [ 103.318547] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.220Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1175gitlab # [ 103.320382] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.222Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1176gitlab # [ 103.322803] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.224Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1177gitlab # [ 103.325024] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.227Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1178gitlab # [ 103.327035] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.229Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1179gitlab # [ 103.328760] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.230Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1180gitlab # [ 103.330842] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.232Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1181gitlab # [ 103.332888] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.234Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1182gitlab # [ 103.335086] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.237Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1183gitlab # [ 103.337456] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.239Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1184gitlab # [ 103.339590] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.241Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1185gitlab # [ 103.341365] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.243Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1186gitlab # [ 103.343686] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.245Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1187gitlab # [ 103.345815] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.247Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1188gitlab # [ 103.347890] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.249Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1189gitlab # [ 103.350131] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.252Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1190gitlab # [ 103.352293] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.254Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1191gitlab # [ 103.360353] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.262Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1192gitlab # [ 103.362664] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.264Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1193gitlab # [ 103.364458] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.266Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1194gitlab # [ 103.366740] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.268Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1195gitlab # [ 103.368922] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.270Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1196gitlab # [ 103.371469] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.273Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1197gitlab # [ 103.373143] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.275Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1198gitlab # [ 103.375181] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.277Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1199gitlab # [ 103.377432] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.279Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1200gitlab # [ 103.379117] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.281Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1201gitlab # [ 103.381641] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.283Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1202gitlab # [ 103.383826] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.285Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1203gitlab # [ 103.385761] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.287Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1204gitlab # [ 103.388133] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.290Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1205gitlab # [ 103.389809] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.291Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1206gitlab # [ 103.391801] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.293Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1207gitlab # [ 103.393632] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.295Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1208gitlab # [ 103.395969] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.297Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1209gitlab # [ 103.398166] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.300Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1210gitlab # [ 103.400359] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.302Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1211gitlab # [ 103.405163] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.307Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1212gitlab # [ 103.410187] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.312Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1213gitlab # [ 103.434489] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.336Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1214gitlab # [ 103.437082] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.337Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1215gitlab # [ 103.437889] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.339Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1216gitlab # [ 103.439950] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.341Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1217gitlab # [ 103.442512] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.344Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1218gitlab # [ 103.444444] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.346Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1219gitlab # [ 103.447131] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.349Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1220gitlab # [ 103.449055] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.351Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1221gitlab # [ 103.452076] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.354Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1222gitlab # [ 103.453853] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.355Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1223gitlab # [ 103.456939] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.358Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1224gitlab # [ 103.458736] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.360Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1225gitlab # [ 103.461747] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.363Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1226gitlab # [ 103.463394] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.365Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1227gitlab # [ 103.465827] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.367Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1228gitlab # [ 103.467722] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.369Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1229gitlab # [ 103.470269] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.372Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1230gitlab # [ 103.472166] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.374Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1231gitlab # [ 103.474174] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.376Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1232gitlab # [ 103.476243] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.378Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1233gitlab # [ 103.479240] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.381Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1234gitlab # [ 103.481237] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.383Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1235gitlab # [ 103.483506] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.385Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1236gitlab # [ 103.485198] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.387Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1237gitlab # [ 103.494539] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.396Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1238gitlab # [ 103.495979] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.397Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1239gitlab # [ 103.498023] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.400Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1240gitlab # [ 103.501478] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.403Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1241gitlab # [ 103.503555] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.405Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1242gitlab # [ 103.505247] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.407Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1243gitlab # [ 103.507633] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.409Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1244gitlab # [ 103.509727] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.411Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1245gitlab # [ 103.511822] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.413Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1246gitlab # [ 103.514333] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.416Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1247gitlab # [ 103.516171] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:24.418Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1248gitlab # [ 108.682761] gitlab-workhorse[1292]: time="2026-07-21T06:35:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1249gitlab # [ 118.688712] gitlab-workhorse[1292]: time="2026-07-21T06:35:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1250gitlab # [ 123.526214] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:44.427Z","message":"Booted Rails 7.2.3 application in production environment"}1251gitlab # [ 123.530642] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:44.428Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1252gitlab # [ 123.535225] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:44.428Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1253gitlab # [ 123.539333] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:44.428Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1254gitlab # [ 123.543919] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:44.431Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1255gitlab # [ 124.500348] bundle[2037]: {"timestamp":"2026-07-21T06:35:45.401Z","pid":2037,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1256gitlab # [ 124.505169] bundle[2037]: {"timestamp":"2026-07-21T06:35:45.401Z","pid":2037,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1257gitlab # [ 124.509208] bundle[2037]: {"timestamp":"2026-07-21T06:35:45.401Z","pid":2037,"message":"! #\u003cThread:0x000073e7d51cb6d8 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1258gitlab # [ 124.520768] bundle[2037]: {"timestamp":"2026-07-21T06:35:45.401Z","pid":2037,"message":"! #\u003cThread:0x000073e7d5402618 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1259gitlab # [ 124.532246] bundle[2037]: {"timestamp":"2026-07-21T06:35:45.401Z","pid":2037,"message":"Use Ctrl-C to stop"}1260gitlab # [ 124.535362] bundle[2037]: {"timestamp":"2026-07-21T06:35:45.402Z","pid":2037,"message":"Sending status to systemd every 1.0 sec"}1261gitlab # [ 124.798043] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:45.699Z","message":"Cleaning working queues"}1262gitlab # [ 124.799829] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:45.700Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b25c9c902fd66be3aea111e8","created_at":"2026-07-21T06:34:53.503Z","trace_propagation_headers":{"sentry-trace":"d6b6674bf3f942bd9d21922120c31cff-61e8c79c7a234bd5","baggage":"sentry-trace_id=d6b6674bf3f942bd9d21922120c31cff,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"987cd0762f33c1348a95a69ae167c9a6","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-21T06:34:53.510Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b25c9c902fd66be3aea111e8: start","job_status":"start","queue_duration_s":52.190113,"scheduling_latency_s":52.19015}1263gitlab # [ 124.845231] bundle[2037]: {"timestamp":"2026-07-21T06:35:45.747Z","pid":2037,"message":"- Worker 0 (PID: 2532) booted in 0.31s, phase: 0"}1264gitlab # [ 124.868756] bundle[2037]: {"timestamp":"2026-07-21T06:35:45.770Z","pid":2037,"message":"- Worker 1 (PID: 2536) booted in 0.31s, phase: 0"}1265gitlab # [ 124.873882] systemd[1]: Started gitlab.service.1266gitlab # [ 124.875894] systemd[1]: Reached target Common target for all GitLab services..1267gitlab # [ 124.881264] systemd[1]: Starting setup-gitlab.service...1268gitlab # [ 124.909370] setup-gitlab-start[2575]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1269gitlab # [ 124.956783] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:35:45.858Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b25c9c902fd66be3aea111e8","created_at":"2026-07-21T06:34:53.503Z","trace_propagation_headers":{"sentry-trace":"d6b6674bf3f942bd9d21922120c31cff-61e8c79c7a234bd5","baggage":"sentry-trace_id=d6b6674bf3f942bd9d21922120c31cff,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"987cd0762f33c1348a95a69ae167c9a6","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-21T06:34:53.510Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b25c9c902fd66be3aea111e8: done: 0.157619 sec","job_status":"done","queue_duration_s":52.190113,"scheduling_latency_s":52.19015,"redis_calls":26,"redis_duration_s":0.003699,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.002004,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000689,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.076218067,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.157619,"completed_at":"2026-07-21T06:35:45.858Z","load_balancing_strategy":"primary","db_duration_s":0.046775,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1270gitlab # [ 127.102073] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:35:48 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 21711271gitlab # [ 127.110209] setup-gitlab-start[2595]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1272gitlab # [ 127.110592] setup-gitlab-start[2596]: ++ jq -r .access_token1273gitlab # [ 127.830976] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:35:48 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 7021274gitlab # [ 127.837866] setup-gitlab-start[2575]: + BEARER=ecd80fa5e3d13bff2e760e97cb1e5952f31281b541ad4ba41ceab74b80b811631275gitlab # [ 127.839295] setup-gitlab-start[2575]: + AUTH='Authorization: Bearer ecd80fa5e3d13bff2e760e97cb1e5952f31281b541ad4ba41ceab74b80b81163'1276gitlab # [ 127.840394] setup-gitlab-start[2575]: + curl -fs -X PUT -H 'Authorization: Bearer ecd80fa5e3d13bff2e760e97cb1e5952f31281b541ad4ba41ceab74b80b81163' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1277gitlab # [ 128.516694] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:35:49 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 6501278gitlab # [ 128.522775] setup-gitlab-start[2575]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1279gitlab # [ 128.539816] runuser[2611]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1280gitlab # [ 128.922426] setup-gitlab-start[2612]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1281gitlab # [ 128.922803] setup-gitlab-start[2612]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1282gitlab # [ 128.924349] setup-gitlab-start[2612]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1283gitlab # [ 129.458176] setup-gitlab-start[2612]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1284gitlab # [ 129.458853] setup-gitlab-start[2612]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1285gitlab # [ 129.461928] setup-gitlab-start[2612]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1286gitlab # [ 141.083112] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:01.983Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"882f969afe386025dd563a78","created_at":"2026-07-21T06:36:01.949Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65c00848f216ed093f9cda23a1aa2c5f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:01.981Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-882f969afe386025dd563a78: start","job_status":"start","queue_duration_s":0.002405,"scheduling_latency_s":0.002452}1287gitlab # [ 141.132466] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"882f969afe386025dd563a78","created_at":"2026-07-21T06:36:01.949Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65c00848f216ed093f9cda23a1aa2c5f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:01.981Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-882f969afe386025dd563a78: done: 0.049928 sec","job_status":"done","queue_duration_s":0.002405,"scheduling_latency_s":0.002452,"redis_calls":12,"redis_duration_s":0.011539,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006972,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002873,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001694,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00085821,"exclusive_lock_hold_duration_s":0.008398553,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.049928,"completed_at":"2026-07-21T06:36:02.033Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1288gitlab # [ 141.175327] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7e7c928d527370efdfac801d","created_at":"2026-07-21T06:36:01.994Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d55ba26a815369419beac76419e393c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.001Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7e7c928d527370efdfac801d: start","job_status":"start","queue_duration_s":0.035793,"scheduling_latency_s":0.035812}1289gitlab # [ 141.175974] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.061Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1290gitlab # [ 141.176633] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7e7c928d527370efdfac801d","created_at":"2026-07-21T06:36:01.994Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d55ba26a815369419beac76419e393c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.001Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7e7c928d527370efdfac801d: done: 0.03653 sec","job_status":"done","queue_duration_s":0.035793,"scheduling_latency_s":0.035812,"redis_calls":10,"redis_duration_s":0.012251,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002224,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001902,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00172927,"duration_s":0.03653,"completed_at":"2026-07-21T06:36:02.073Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00131,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1291gitlab # [ 141.215672] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"acf529beac629512c4d7c5bf","created_at":"2026-07-21T06:36:02.014Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f5d3390e57f3c539c1bd087ebf88f6a","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.040Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-acf529beac629512c4d7c5bf: start","job_status":"start","queue_duration_s":0.034573,"scheduling_latency_s":0.034586}1292gitlab # [ 141.325902] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"acf529beac629512c4d7c5bf","created_at":"2026-07-21T06:36:02.014Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f5d3390e57f3c539c1bd087ebf88f6a","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.040Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-acf529beac629512c4d7c5bf: done: 0.150234 sec","job_status":"done","queue_duration_s":0.034573,"scheduling_latency_s":0.034586,"redis_calls":10,"redis_duration_s":0.011257,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003647,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006063,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001547,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00096772,"extra.ci_runners_stale_machines_cleanup_cron_worker.status":"success","extra.ci_runners_stale_machines_cleanup_cron_worker.total_deleted":0,"extra.ci_runners_stale_machines_cleanup_cron_worker.batch_counts":[0],"duration_s":0.150234,"completed_at":"2026-07-21T06:36:02.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1293gitlab # [ 141.367747] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"0b6699ad12ab22f5133ea4b0","created_at":"2026-07-21T06:36:02.056Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.068Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-0b6699ad12ab22f5133ea4b0: start","job_status":"start","queue_duration_s":0.200793,"scheduling_latency_s":0.200824}1294gitlab # [ 141.484865] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.386Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"0b6699ad12ab22f5133ea4b0","created_at":"2026-07-21T06:36:02.056Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.068Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-0b6699ad12ab22f5133ea4b0: done: 0.116711 sec","job_status":"done","queue_duration_s":0.200793,"scheduling_latency_s":0.200824,"redis_calls":19,"redis_duration_s":0.024823,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004256,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000768,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.01332,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006479,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001347936,"duration_s":0.116711,"completed_at":"2026-07-21T06:36:02.385Z","load_balancing_strategy":"primary","db_duration_s":0.00429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1295gitlab # [ 141.548747] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.393Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"f88ec763074266e87189eeb2","created_at":"2026-07-21T06:36:02.163Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ce7a6447a0ace8aaa9740ac5ffb08dc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.180Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-f88ec763074266e87189eeb2: start","job_status":"start","queue_duration_s":0.212725,"scheduling_latency_s":0.21276}1296gitlab # [ 141.561542] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.432Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"f88ec763074266e87189eeb2","created_at":"2026-07-21T06:36:02.163Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ce7a6447a0ace8aaa9740ac5ffb08dc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.180Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-f88ec763074266e87189eeb2: done: 0.039298 sec","job_status":"done","queue_duration_s":0.212725,"scheduling_latency_s":0.21276,"redis_calls":12,"redis_duration_s":0.003776,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000754,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000652876,"exclusive_lock_hold_duration_s":0.010478425999999999,"duration_s":0.039298,"completed_at":"2026-07-21T06:36:02.432Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1297gitlab # [ 141.599709] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.438Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"eafec602f4107833510b63cf","created_at":"2026-07-21T06:36:02.194Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6066ba76371aa3ea431c250ab5c2ecb8","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.201Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-eafec602f4107833510b63cf: start","job_status":"start","queue_duration_s":0.236677,"scheduling_latency_s":0.236704}1298gitlab # [ 142.010100] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"eafec602f4107833510b63cf","created_at":"2026-07-21T06:36:02.194Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6066ba76371aa3ea431c250ab5c2ecb8","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.201Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-eafec602f4107833510b63cf: done: 0.473315 sec","job_status":"done","queue_duration_s":0.236677,"scheduling_latency_s":0.236704,"redis_calls":574,"redis_duration_s":0.22986,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002241,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.223978,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.223694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346413,"duration_s":0.473315,"completed_at":"2026-07-21T06:36:02.911Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044748,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1299gitlab # [ 142.048380] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.913Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f94dbe83b328ff39d778a2b0","created_at":"2026-07-21T06:36:02.219Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac2f14a5e98ffbd32490d644522159da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.231Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f94dbe83b328ff39d778a2b0: start","job_status":"start","queue_duration_s":0.682188,"scheduling_latency_s":0.682213}1300gitlab # [ 142.061761] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.925Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1301gitlab # [ 142.065351] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.930Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f94dbe83b328ff39d778a2b0","created_at":"2026-07-21T06:36:02.219Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac2f14a5e98ffbd32490d644522159da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.231Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f94dbe83b328ff39d778a2b0: done: 0.016826 sec","job_status":"done","queue_duration_s":0.682188,"scheduling_latency_s":0.682213,"redis_calls":10,"redis_duration_s":0.002185,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001229,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000268749,"duration_s":0.016826,"completed_at":"2026-07-21T06:36:02.930Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000606,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 142.103556] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:02.931Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6ad5c8eba78f953ec3d52232","created_at":"2026-07-21T06:36:02.301Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.309Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6ad5c8eba78f953ec3d52232: start","job_status":"start","queue_duration_s":0.622324,"scheduling_latency_s":0.622338}1303gitlab # [ 142.115368] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:03.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6ad5c8eba78f953ec3d52232","created_at":"2026-07-21T06:36:02.301Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.309Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6ad5c8eba78f953ec3d52232: done: 0.070642 sec","job_status":"done","queue_duration_s":0.622324,"scheduling_latency_s":0.622338,"redis_calls":71,"redis_duration_s":0.015863,"redis_read_bytes":120,"redis_write_bytes":42524,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002165,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001049,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37335,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.012052,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000597,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002992,"duration_s":0.070642,"completed_at":"2026-07-21T06:36:03.002Z","load_balancing_strategy":"primary","db_duration_s":0.001847,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 142.152190] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:03.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9fcdf117ef4c1d0d4e4be369","created_at":"2026-07-21T06:36:02.314Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3f0e7044979bd504ee1f3618e82af1a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.329Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9fcdf117ef4c1d0d4e4be369: start","job_status":"start","queue_duration_s":0.674253,"scheduling_latency_s":0.674267}1305gitlab # [ 142.164153] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:03.023Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9fcdf117ef4c1d0d4e4be369","created_at":"2026-07-21T06:36:02.314Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3f0e7044979bd504ee1f3618e82af1a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.329Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9fcdf117ef4c1d0d4e4be369: done: 0.020051 sec","job_status":"done","queue_duration_s":0.674253,"scheduling_latency_s":0.674267,"redis_calls":12,"redis_duration_s":0.002682,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001254,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000370159,"exclusive_lock_hold_duration_s":0.0062111250000000005,"duration_s":0.020051,"completed_at":"2026-07-21T06:36:03.023Z","load_balancing_strategy":"primary","db_duration_s":0.00442,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 142.164675] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:03.025Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"23c2e3a790471a78ad5dd780","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77b5b6a6483ce8076719b58b4b8366a1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.359Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-23c2e3a790471a78ad5dd780: start","job_status":"start","queue_duration_s":0.665557,"scheduling_latency_s":0.66557}1307gitlab # [ 142.174140] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:03.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"23c2e3a790471a78ad5dd780","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77b5b6a6483ce8076719b58b4b8366a1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.359Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-23c2e3a790471a78ad5dd780: done: 0.024285 sec","job_status":"done","queue_duration_s":0.665557,"scheduling_latency_s":0.66557,"redis_calls":10,"redis_duration_s":0.002265,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001442,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000260927,"duration_s":0.024285,"completed_at":"2026-07-21T06:36:03.049Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003369,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 142.213074] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:03.051Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"81e35fcb24ef1796da8d589e","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5d8bb9c9ed7e4552","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.369Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-81e35fcb24ef1796da8d589e: start","job_status":"start","queue_duration_s":0.681803,"scheduling_latency_s":0.681841}1309gitlab # [ 143.400738] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.302Z","message":"No existing merge request to be cleaned up."}1310gitlab # [ 143.430605] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.331Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"81e35fcb24ef1796da8d589e","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5d8bb9c9ed7e4552","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.369Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-81e35fcb24ef1796da8d589e: done: 1.280419 sec","job_status":"done","queue_duration_s":0.681803,"scheduling_latency_s":0.681841,"redis_calls":13,"redis_duration_s":0.1372,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.099896,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.036083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.303978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000429943,"duration_s":1.280419,"completed_at":"2026-07-21T06:36:04.331Z","load_balancing_strategy":"primary","db_duration_s":0.012947,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 143.487851] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.389Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fc0b680261f37e863cfb61d2","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5d8bb9c9ed7e4552","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.369Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fc0b680261f37e863cfb61d2: start","job_status":"start","queue_duration_s":2.019981,"scheduling_latency_s":2.020008}1312gitlab # [ 143.511683] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.413Z","message":"No existing merge request to be cleaned up."}1313gitlab # [ 143.539610] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.439Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fc0b680261f37e863cfb61d2","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5d8bb9c9ed7e4552","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.369Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fc0b680261f37e863cfb61d2: done: 0.050078 sec","job_status":"done","queue_duration_s":2.019981,"scheduling_latency_s":2.020008,"redis_calls":10,"redis_duration_s":0.011723,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006126,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005597,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.050078,"completed_at":"2026-07-21T06:36:04.439Z","load_balancing_strategy":"primary","db_duration_s":0.008979,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 143.580781] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.482Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"85f4735c39deb8ccf075f559","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5d8bb9c9ed7e4552","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.369Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-85f4735c39deb8ccf075f559: start","job_status":"start","queue_duration_s":2.112857,"scheduling_latency_s":2.11289}1315gitlab # [ 143.652304] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.553Z","message":"No existing merge request to be cleaned up."}1316gitlab # [ 143.682582] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.583Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"85f4735c39deb8ccf075f559","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5d8bb9c9ed7e4552","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.369Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-85f4735c39deb8ccf075f559: done: 0.101402 sec","job_status":"done","queue_duration_s":2.112857,"scheduling_latency_s":2.11289,"redis_calls":10,"redis_duration_s":0.012654,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006002,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.006652,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.101402,"completed_at":"2026-07-21T06:36:04.583Z","load_balancing_strategy":"primary","db_duration_s":0.010734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 143.722572] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.592Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a225ea066924124c26fd263e","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5d8bb9c9ed7e4552","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.369Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a225ea066924124c26fd263e: start","job_status":"start","queue_duration_s":2.22263,"scheduling_latency_s":2.222667}1318gitlab # [ 143.747853] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.649Z","message":"No existing merge request to be cleaned up."}1319gitlab # [ 143.767087] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.668Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a225ea066924124c26fd263e","created_at":"2026-07-21T06:36:02.343Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5d8bb9c9ed7e4552","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"55d8d4e2b2feb90cbe115508d6147a88","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.369Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a225ea066924124c26fd263e: done: 0.076007 sec","job_status":"done","queue_duration_s":2.22263,"scheduling_latency_s":2.222667,"redis_calls":10,"redis_duration_s":0.011559,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004351,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.007208,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.031,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.031,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.076007,"completed_at":"2026-07-21T06:36:04.668Z","load_balancing_strategy":"primary","db_duration_s":0.037286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 143.808247] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3da687f8a57c886a21b29ad0","created_at":"2026-07-21T06:36:02.370Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a59583ee9cdc9af04441c92bc5cd7d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-21T06:36:02.383Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3da687f8a57c886a21b29ad0: start","job_status":"start","queue_duration_s":2.325828,"scheduling_latency_s":2.32587}1321gitlab # [ 143.847370] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.749Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1322gitlab # [ 143.866253] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.767Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3da687f8a57c886a21b29ad0","created_at":"2026-07-21T06:36:02.370Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a59583ee9cdc9af04441c92bc5cd7d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-21T06:36:02.383Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3da687f8a57c886a21b29ad0: done: 0.058176 sec","job_status":"done","queue_duration_s":2.325828,"scheduling_latency_s":2.32587,"redis_calls":10,"redis_duration_s":0.011348,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006968,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002964,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001589867,"duration_s":0.058176,"completed_at":"2026-07-21T06:36:04.767Z","load_balancing_strategy":"primary","db_duration_s":0.000904,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 143.907526] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.808Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"dd568d9a15b01e7680d19eed","created_at":"2026-07-21T06:36:02.393Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a927583ef95e5240d9b23b2501a5a79b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.437Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-dd568d9a15b01e7680d19eed: start","job_status":"start","queue_duration_s":2.37125,"scheduling_latency_s":2.371294}1324gitlab # [ 143.948117] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.849Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"dd568d9a15b01e7680d19eed","created_at":"2026-07-21T06:36:02.393Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a927583ef95e5240d9b23b2501a5a79b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.437Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-dd568d9a15b01e7680d19eed: done: 0.040688 sec","job_status":"done","queue_duration_s":2.37125,"scheduling_latency_s":2.371294,"redis_calls":12,"redis_duration_s":0.010092,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.002124,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00125,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00069925,"duration_s":0.040688,"completed_at":"2026-07-21T06:36:04.849Z","load_balancing_strategy":"primary","db_duration_s":0.001986,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 143.969691] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.857Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7244cd5e747d742b46f01438","created_at":"2026-07-21T06:36:02.465Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"07d0e100c47ac17e131f87423e179cf7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.476Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7244cd5e747d742b46f01438: start","job_status":"start","queue_duration_s":2.381015,"scheduling_latency_s":2.381055}1326gitlab # [ 144.088392] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:04.989Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7244cd5e747d742b46f01438","created_at":"2026-07-21T06:36:02.465Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"07d0e100c47ac17e131f87423e179cf7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.476Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7244cd5e747d742b46f01438: done: 0.132663 sec","job_status":"done","queue_duration_s":2.381015,"scheduling_latency_s":2.381055,"redis_calls":12,"redis_duration_s":0.042865,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002039,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007972,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.032854,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000542248,"exclusive_lock_hold_duration_s":0.039662583,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.132663,"completed_at":"2026-07-21T06:36:04.989Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027925,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 144.140458] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.042Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f1479c3cd9eb606235bee0cd","created_at":"2026-07-21T06:36:02.479Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5030bbb883c3b02c059288b3d141aaa","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.484Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f1479c3cd9eb606235bee0cd: start","job_status":"start","queue_duration_s":2.558088,"scheduling_latency_s":2.558112}1328gitlab # [ 144.220711] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f1479c3cd9eb606235bee0cd","created_at":"2026-07-21T06:36:02.479Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5030bbb883c3b02c059288b3d141aaa","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.484Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f1479c3cd9eb606235bee0cd: done: 0.07989 sec","job_status":"done","queue_duration_s":2.558088,"scheduling_latency_s":2.558112,"redis_calls":16,"redis_duration_s":0.021308,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.010343,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00852,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002445,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00086687,"duration_s":0.07989,"completed_at":"2026-07-21T06:36:05.122Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0187,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 144.256170] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.130Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"976cda97a19964953d2adcac","created_at":"2026-07-21T06:36:02.488Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"faa1e8b06f73912a678054bc90345270","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.491Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-976cda97a19964953d2adcac: start","job_status":"start","queue_duration_s":2.638659,"scheduling_latency_s":2.638688}1330gitlab # [ 144.390561] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.291Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"976cda97a19964953d2adcac","created_at":"2026-07-21T06:36:02.488Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"faa1e8b06f73912a678054bc90345270","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.491Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-976cda97a19964953d2adcac: done: 0.160817 sec","job_status":"done","queue_duration_s":2.638659,"scheduling_latency_s":2.638688,"redis_calls":14,"redis_duration_s":0.017306000000000002,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002945,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00804,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006321,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000725512,"exclusive_lock_hold_duration_s":0.110254237,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.160817,"completed_at":"2026-07-21T06:36:05.291Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.110836,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 144.431148] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8830488b2be9c1510837b060","created_at":"2026-07-21T06:36:02.494Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1380f1de8efec59fcecdbfb5f3874ce5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-21T06:36:02.496Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8830488b2be9c1510837b060: start","job_status":"start","queue_duration_s":2.800849,"scheduling_latency_s":2.800889}1332gitlab # [ 144.432073] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.328Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1333gitlab # [ 144.474520] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8830488b2be9c1510837b060","created_at":"2026-07-21T06:36:02.494Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1380f1de8efec59fcecdbfb5f3874ce5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-21T06:36:02.496Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8830488b2be9c1510837b060: done: 0.078329 sec","job_status":"done","queue_duration_s":2.800849,"scheduling_latency_s":2.800889,"redis_calls":10,"redis_duration_s":0.046110000000000005,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003079,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.041555,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001476,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005031924,"duration_s":0.078329,"completed_at":"2026-07-21T06:36:05.375Z","load_balancing_strategy":"primary","db_duration_s":0.002729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 144.475927] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.377Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"fd6d9f451509175c96898490","created_at":"2026-07-21T06:36:02.503Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eee9fad78a1992bde5bdabc4f4c134ac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.508Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-fd6d9f451509175c96898490: start","job_status":"start","queue_duration_s":2.868139,"scheduling_latency_s":2.868154}1335gitlab # [ 144.539697] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.440Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"fd6d9f451509175c96898490","created_at":"2026-07-21T06:36:02.503Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eee9fad78a1992bde5bdabc4f4c134ac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.508Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-fd6d9f451509175c96898490: done: 0.063812 sec","job_status":"done","queue_duration_s":2.868139,"scheduling_latency_s":2.868154,"redis_calls":12,"redis_duration_s":0.001588,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000589,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000194717,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.063812,"completed_at":"2026-07-21T06:36:05.440Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000409,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 144.540985] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"0c27ff987fc2e65a092f60c1","created_at":"2026-07-21T06:36:02.512Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66fd2069ff6d20c65dc13aa3b4d7b207","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.516Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-0c27ff987fc2e65a092f60c1: start","job_status":"start","queue_duration_s":2.925897,"scheduling_latency_s":2.925911}1337gitlab # [ 144.592108] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"0c27ff987fc2e65a092f60c1","created_at":"2026-07-21T06:36:02.512Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66fd2069ff6d20c65dc13aa3b4d7b207","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.516Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-0c27ff987fc2e65a092f60c1: done: 0.02207 sec","job_status":"done","queue_duration_s":2.925897,"scheduling_latency_s":2.925911,"redis_calls":20,"redis_duration_s":0.0024980000000000002,"redis_read_bytes":20,"redis_write_bytes":3248,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001054,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1346,"redis_queues_calls":2,"redis_queues_duration_s":0.000207,"redis_queues_read_bytes":3,"redis_queues_write_bytes":818,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020254,"duration_s":0.02207,"completed_at":"2026-07-21T06:36:05.464Z","load_balancing_strategy":"primary","db_duration_s":0.000863,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 144.592808] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.465Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"4c9c2e5cffb0548698e9e6fd","created_at":"2026-07-21T06:36:02.531Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"31bb8659a0be6245ed209cb983b7b5d0","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.536Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-4c9c2e5cffb0548698e9e6fd: start","job_status":"start","queue_duration_s":2.928795,"scheduling_latency_s":2.928809}1339gitlab # [ 144.603527] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.477Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"4c9c2e5cffb0548698e9e6fd","created_at":"2026-07-21T06:36:02.531Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"31bb8659a0be6245ed209cb983b7b5d0","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.536Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-4c9c2e5cffb0548698e9e6fd: done: 0.012604 sec","job_status":"done","queue_duration_s":2.928795,"scheduling_latency_s":2.928809,"redis_calls":10,"redis_duration_s":0.001277,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197511,"duration_s":0.012604,"completed_at":"2026-07-21T06:36:05.477Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 144.653023] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.478Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"464b31fd58c3eb602f93cc54","created_at":"2026-07-21T06:36:02.565Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1e751e534351dce387db1bd0bf17f95","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.574Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-464b31fd58c3eb602f93cc54: start","job_status":"start","queue_duration_s":2.903927,"scheduling_latency_s":2.90394}1341gitlab # [ 144.653742] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.495Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"464b31fd58c3eb602f93cc54","created_at":"2026-07-21T06:36:02.565Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1e751e534351dce387db1bd0bf17f95","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.574Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-464b31fd58c3eb602f93cc54: done: 0.016205 sec","job_status":"done","queue_duration_s":2.903927,"scheduling_latency_s":2.90394,"redis_calls":10,"redis_duration_s":0.001245,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000189968,"duration_s":0.016205,"completed_at":"2026-07-21T06:36:05.495Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 144.696926] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.495Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"fd37187358d46fd89c1212a3","created_at":"2026-07-21T06:36:02.585Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db6ea6244250353a05883caa36bc693","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.594Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-fd37187358d46fd89c1212a3: start","job_status":"start","queue_duration_s":2.90159,"scheduling_latency_s":2.901603}1343gitlab # [ 144.702880] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"fd37187358d46fd89c1212a3","created_at":"2026-07-21T06:36:02.585Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db6ea6244250353a05883caa36bc693","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.594Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-fd37187358d46fd89c1212a3: done: 0.02172 sec","job_status":"done","queue_duration_s":2.90159,"scheduling_latency_s":2.901603,"redis_calls":13,"redis_duration_s":0.001677,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000555,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007645665,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.02172,"completed_at":"2026-07-21T06:36:05.517Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002269,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1344gitlab # [ 144.747754] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9ea46ddd616815dec3209beb","created_at":"2026-07-21T06:36:02.600Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59c33acc2e41e6873fb50cef4cdc1a21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.606Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9ea46ddd616815dec3209beb: start","job_status":"start","queue_duration_s":2.91251,"scheduling_latency_s":2.912524}1345gitlab # [ 144.790839] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9ea46ddd616815dec3209beb","created_at":"2026-07-21T06:36:02.600Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59c33acc2e41e6873fb50cef4cdc1a21","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.606Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9ea46ddd616815dec3209beb: done: 0.012779 sec","job_status":"done","queue_duration_s":2.91251,"scheduling_latency_s":2.912524,"redis_calls":12,"redis_duration_s":0.001681,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000653,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021567,"duration_s":0.012779,"completed_at":"2026-07-21T06:36:05.531Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 144.791931] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.532Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b03349ac1c689f045e702df0","created_at":"2026-07-21T06:36:02.630Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66f15a8fa6b644dd694305901a8c36d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.638Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b03349ac1c689f045e702df0: start","job_status":"start","queue_duration_s":2.894202,"scheduling_latency_s":2.894214}1347gitlab # [ 144.804765] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.542Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b03349ac1c689f045e702df0","created_at":"2026-07-21T06:36:02.630Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66f15a8fa6b644dd694305901a8c36d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.638Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b03349ac1c689f045e702df0: done: 0.010287 sec","job_status":"done","queue_duration_s":2.894202,"scheduling_latency_s":2.894214,"redis_calls":10,"redis_duration_s":0.001312,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000241,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00019332,"duration_s":0.010287,"completed_at":"2026-07-21T06:36:05.542Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000228,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 144.860664] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.543Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"330ff70c21343dc30d29d19a","created_at":"2026-07-21T06:36:02.644Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"444a6f33bfb5549a880af072971d63ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-21T06:36:02.651Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-330ff70c21343dc30d29d19a: start","job_status":"start","queue_duration_s":2.892069,"scheduling_latency_s":2.892081}1349gitlab # [ 144.861548] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"330ff70c21343dc30d29d19a","created_at":"2026-07-21T06:36:02.644Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"444a6f33bfb5549a880af072971d63ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-21T06:36:02.651Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-330ff70c21343dc30d29d19a: done: 0.016988 sec","job_status":"done","queue_duration_s":2.892069,"scheduling_latency_s":2.892081,"redis_calls":10,"redis_duration_s":0.001261,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222933,"duration_s":0.016988,"completed_at":"2026-07-21T06:36:05.560Z","load_balancing_strategy":"primary","db_duration_s":0.002032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 144.914864] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"96565d72dce867e52280d26b","created_at":"2026-07-21T06:36:02.658Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7fd583ff10576f11231e9e4d9a49af1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.664Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-96565d72dce867e52280d26b: start","job_status":"start","queue_duration_s":2.896801,"scheduling_latency_s":2.896814}1351gitlab # [ 144.915495] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"96565d72dce867e52280d26b","created_at":"2026-07-21T06:36:02.658Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7fd583ff10576f11231e9e4d9a49af1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.664Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-96565d72dce867e52280d26b: done: 0.019305 sec","job_status":"done","queue_duration_s":2.896801,"scheduling_latency_s":2.896814,"redis_calls":10,"redis_duration_s":0.001408,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000244,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000832,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239975,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.019305,"completed_at":"2026-07-21T06:36:05.580Z","load_balancing_strategy":"primary","db_duration_s":0.007309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 144.955600] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7277cedb95ac5fd451f44b5c","created_at":"2026-07-21T06:36:02.668Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"078dac57352796509a0b2b0467a6127c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.673Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7277cedb95ac5fd451f44b5c: start","job_status":"start","queue_duration_s":2.907917,"scheduling_latency_s":2.907929}1353gitlab # [ 144.956192] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.588Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1354gitlab # [ 144.957222] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7277cedb95ac5fd451f44b5c","created_at":"2026-07-21T06:36:02.668Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"078dac57352796509a0b2b0467a6127c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.673Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7277cedb95ac5fd451f44b5c: done: 0.010798 sec","job_status":"done","queue_duration_s":2.907917,"scheduling_latency_s":2.907929,"redis_calls":10,"redis_duration_s":0.001249,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020254,"duration_s":0.010798,"completed_at":"2026-07-21T06:36:05.592Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000392,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 145.002305] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"598a56d0b346c68b265e15db","created_at":"2026-07-21T06:36:02.680Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fd9d480c39a82588b70e83f0df63136","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.685Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-598a56d0b346c68b265e15db: start","job_status":"start","queue_duration_s":2.907891,"scheduling_latency_s":2.907904}1356gitlab # [ 145.007885] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.599Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1357gitlab # [ 145.010848] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.603Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"598a56d0b346c68b265e15db","created_at":"2026-07-21T06:36:02.680Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fd9d480c39a82588b70e83f0df63136","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.685Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-598a56d0b346c68b265e15db: done: 0.01004 sec","job_status":"done","queue_duration_s":2.907891,"scheduling_latency_s":2.907904,"redis_calls":10,"redis_duration_s":0.0012699999999999999,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000190527,"duration_s":0.01004,"completed_at":"2026-07-21T06:36:05.603Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 145.060324] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2b9eeb91709923d3cb741bf0","created_at":"2026-07-21T06:36:02.690Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8b8a6090c80adffef3151b79ba490d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.694Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2b9eeb91709923d3cb741bf0: start","job_status":"start","queue_duration_s":2.909448,"scheduling_latency_s":2.90946}1359gitlab # [ 145.061629] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.738Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"2b9eeb91709923d3cb741bf0","created_at":"2026-07-21T06:36:02.690Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-5c6cdb46b4d044fc","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8b8a6090c80adffef3151b79ba490d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.694Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-2b9eeb91709923d3cb741bf0: done: 0.134616 sec","job_status":"done","queue_duration_s":2.909448,"scheduling_latency_s":2.90946,"redis_calls":14,"redis_duration_s":0.0019130000000000002,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000518,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000648,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00029473,"exclusive_lock_hold_duration_s":0.120464193,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.134616,"completed_at":"2026-07-21T06:36:05.738Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 145.117189] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddc3233aff64d6a24ab4fefd","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddc3233aff64d6a24ab4fefd: start","job_status":"start","queue_duration_s":2.74427,"scheduling_latency_s":2.744284}1361gitlab # [ 145.117871] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddc3233aff64d6a24ab4fefd","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddc3233aff64d6a24ab4fefd: done: 0.013165 sec","job_status":"done","queue_duration_s":2.74427,"scheduling_latency_s":2.744284,"redis_calls":16,"redis_duration_s":0.002019,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001186,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212318,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013165,"completed_at":"2026-07-21T06:36:05.752Z","load_balancing_strategy":"primary","db_duration_s":0.000241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 145.161075] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce44fae91bc15b9d4ad56d57","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce44fae91bc15b9d4ad56d57: start","job_status":"start","queue_duration_s":2.758263,"scheduling_latency_s":2.758276}1363gitlab # [ 145.161781] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce44fae91bc15b9d4ad56d57","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce44fae91bc15b9d4ad56d57: done: 0.00896 sec","job_status":"done","queue_duration_s":2.758263,"scheduling_latency_s":2.758276,"redis_calls":13,"redis_duration_s":0.001626,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00896,"completed_at":"2026-07-21T06:36:05.762Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 145.203694] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"588bc75b5b389e196da0c63b","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-588bc75b5b389e196da0c63b: start","job_status":"start","queue_duration_s":2.76802,"scheduling_latency_s":2.768033}1365gitlab # [ 145.204441] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"588bc75b5b389e196da0c63b","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-588bc75b5b389e196da0c63b: done: 0.008869 sec","job_status":"done","queue_duration_s":2.76802,"scheduling_latency_s":2.768033,"redis_calls":13,"redis_duration_s":0.001553,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008869,"completed_at":"2026-07-21T06:36:05.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 145.241786] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"469120ce157733942a8efe15","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-469120ce157733942a8efe15: start","job_status":"start","queue_duration_s":2.777703,"scheduling_latency_s":2.777716}1367gitlab # [ 145.248767] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"469120ce157733942a8efe15","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-469120ce157733942a8efe15: done: 0.008834 sec","job_status":"done","queue_duration_s":2.777703,"scheduling_latency_s":2.777716,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008834,"completed_at":"2026-07-21T06:36:05.781Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 145.283571] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64fbb61d2bbad2daf0752de5","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64fbb61d2bbad2daf0752de5: start","job_status":"start","queue_duration_s":2.787318,"scheduling_latency_s":2.787332}1369gitlab # [ 145.322781] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64fbb61d2bbad2daf0752de5","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64fbb61d2bbad2daf0752de5: done: 0.008621 sec","job_status":"done","queue_duration_s":2.787318,"scheduling_latency_s":2.787332,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008621,"completed_at":"2026-07-21T06:36:05.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 145.327991] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f22fa8f68bd1210c1f213aa","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f22fa8f68bd1210c1f213aa: start","job_status":"start","queue_duration_s":2.796787,"scheduling_latency_s":2.7968}1371gitlab # [ 145.339115] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f22fa8f68bd1210c1f213aa","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f22fa8f68bd1210c1f213aa: done: 0.008713 sec","job_status":"done","queue_duration_s":2.796787,"scheduling_latency_s":2.7968,"redis_calls":13,"redis_duration_s":0.00157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008713,"completed_at":"2026-07-21T06:36:05.800Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 145.409875] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7a454b9a24a6a7804e687b9","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7a454b9a24a6a7804e687b9: start","job_status":"start","queue_duration_s":2.806318,"scheduling_latency_s":2.806331}1373gitlab # [ 145.410645] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7a454b9a24a6a7804e687b9","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7a454b9a24a6a7804e687b9: done: 0.008649 sec","job_status":"done","queue_duration_s":2.806318,"scheduling_latency_s":2.806331,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008649,"completed_at":"2026-07-21T06:36:05.810Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 145.451078] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8f15e503ce0db638244743e","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8f15e503ce0db638244743e: start","job_status":"start","queue_duration_s":2.815768,"scheduling_latency_s":2.815781}1375gitlab # [ 145.451796] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8f15e503ce0db638244743e","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8f15e503ce0db638244743e: done: 0.008716 sec","job_status":"done","queue_duration_s":2.815768,"scheduling_latency_s":2.815781,"redis_calls":13,"redis_duration_s":0.001547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008716,"completed_at":"2026-07-21T06:36:05.819Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 145.463221] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d0d48511e6b4c188b5aa885","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d0d48511e6b4c188b5aa885: start","job_status":"start","queue_duration_s":2.825313,"scheduling_latency_s":2.825327}1377gitlab # [ 145.474456] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d0d48511e6b4c188b5aa885","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d0d48511e6b4c188b5aa885: done: 0.009063 sec","job_status":"done","queue_duration_s":2.825313,"scheduling_latency_s":2.825327,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009063,"completed_at":"2026-07-21T06:36:05.829Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 145.511785] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1017969422b0cfbada261dc5","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1017969422b0cfbada261dc5: start","job_status":"start","queue_duration_s":2.835178,"scheduling_latency_s":2.835192}1379gitlab # [ 145.519020] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1017969422b0cfbada261dc5","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1017969422b0cfbada261dc5: done: 0.008696 sec","job_status":"done","queue_duration_s":2.835178,"scheduling_latency_s":2.835192,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008696,"completed_at":"2026-07-21T06:36:05.839Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 145.574931] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a4f801e97e62ea3acc5ddd4","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a4f801e97e62ea3acc5ddd4: start","job_status":"start","queue_duration_s":2.844649,"scheduling_latency_s":2.844686}1381gitlab # [ 145.575741] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a4f801e97e62ea3acc5ddd4","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a4f801e97e62ea3acc5ddd4: done: 0.00862 sec","job_status":"done","queue_duration_s":2.844649,"scheduling_latency_s":2.844686,"redis_calls":13,"redis_duration_s":0.0015069999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00862,"completed_at":"2026-07-21T06:36:05.848Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 145.627240] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"611784e5db56e01bbc6dbc7c","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-611784e5db56e01bbc6dbc7c: start","job_status":"start","queue_duration_s":2.85408,"scheduling_latency_s":2.854093}1383gitlab # [ 145.627791] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"611784e5db56e01bbc6dbc7c","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-611784e5db56e01bbc6dbc7c: done: 0.008737 sec","job_status":"done","queue_duration_s":2.85408,"scheduling_latency_s":2.854093,"redis_calls":13,"redis_duration_s":0.0015940000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008737,"completed_at":"2026-07-21T06:36:05.858Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 145.670848] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f388badd8cd39a8cd91c377d","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f388badd8cd39a8cd91c377d: start","job_status":"start","queue_duration_s":2.863602,"scheduling_latency_s":2.863614}1385gitlab # [ 145.671587] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f388badd8cd39a8cd91c377d","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f388badd8cd39a8cd91c377d: done: 0.008672 sec","job_status":"done","queue_duration_s":2.863602,"scheduling_latency_s":2.863614,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008672,"completed_at":"2026-07-21T06:36:05.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 145.712259] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b7129b9b0490a15fc0dd5c0","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b7129b9b0490a15fc0dd5c0: start","job_status":"start","queue_duration_s":2.87306,"scheduling_latency_s":2.873072}1387gitlab # [ 145.712982] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b7129b9b0490a15fc0dd5c0","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b7129b9b0490a15fc0dd5c0: done: 0.008668 sec","job_status":"done","queue_duration_s":2.87306,"scheduling_latency_s":2.873072,"redis_calls":13,"redis_duration_s":0.0015439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008668,"completed_at":"2026-07-21T06:36:05.877Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 145.752068] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"927a56a301436654942445f5","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-927a56a301436654942445f5: start","job_status":"start","queue_duration_s":2.882517,"scheduling_latency_s":2.88253}1389gitlab # [ 145.752705] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"927a56a301436654942445f5","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-927a56a301436654942445f5: done: 0.008659 sec","job_status":"done","queue_duration_s":2.882517,"scheduling_latency_s":2.88253,"redis_calls":13,"redis_duration_s":0.001492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008659,"completed_at":"2026-07-21T06:36:05.886Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 145.807877] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b023557750f8ba6206ae5a2","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b023557750f8ba6206ae5a2: start","job_status":"start","queue_duration_s":2.891974,"scheduling_latency_s":2.891987}1391gitlab # [ 145.808626] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b023557750f8ba6206ae5a2","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b023557750f8ba6206ae5a2: done: 0.008777 sec","job_status":"done","queue_duration_s":2.891974,"scheduling_latency_s":2.891987,"redis_calls":13,"redis_duration_s":0.001557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008777,"completed_at":"2026-07-21T06:36:05.896Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 145.819893] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f025921b18ae62cd97b3aa09","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f025921b18ae62cd97b3aa09: start","job_status":"start","queue_duration_s":2.901528,"scheduling_latency_s":2.901541}1393gitlab # [ 145.831047] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f025921b18ae62cd97b3aa09","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f025921b18ae62cd97b3aa09: done: 0.008664 sec","job_status":"done","queue_duration_s":2.901528,"scheduling_latency_s":2.901541,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008664,"completed_at":"2026-07-21T06:36:05.905Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 145.865900] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0cb7bab78ba42af2aafd819","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0cb7bab78ba42af2aafd819: start","job_status":"start","queue_duration_s":2.911043,"scheduling_latency_s":2.911057}1395gitlab # [ 145.908599] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0cb7bab78ba42af2aafd819","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0cb7bab78ba42af2aafd819: done: 0.008701 sec","job_status":"done","queue_duration_s":2.911043,"scheduling_latency_s":2.911057,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008701,"completed_at":"2026-07-21T06:36:05.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 145.910671] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dc42e6ff2b02070a0f1edff","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dc42e6ff2b02070a0f1edff: start","job_status":"start","queue_duration_s":2.920538,"scheduling_latency_s":2.920551}1397gitlab # [ 145.921875] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dc42e6ff2b02070a0f1edff","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dc42e6ff2b02070a0f1edff: done: 0.008978 sec","job_status":"done","queue_duration_s":2.920538,"scheduling_latency_s":2.920551,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008978,"completed_at":"2026-07-21T06:36:05.924Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 145.985123] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20bf67aeb3b9aa59e479f836","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20bf67aeb3b9aa59e479f836: start","job_status":"start","queue_duration_s":2.930332,"scheduling_latency_s":2.930345}1399gitlab # [ 145.985836] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20bf67aeb3b9aa59e479f836","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20bf67aeb3b9aa59e479f836: done: 0.009503 sec","job_status":"done","queue_duration_s":2.930332,"scheduling_latency_s":2.930345,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009503,"completed_at":"2026-07-21T06:36:05.935Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 146.039136] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"159be2f57fbadb266eac4649","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-159be2f57fbadb266eac4649: start","job_status":"start","queue_duration_s":2.940621,"scheduling_latency_s":2.940634}1401gitlab # [ 146.039864] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"159be2f57fbadb266eac4649","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-159be2f57fbadb266eac4649: done: 0.00916 sec","job_status":"done","queue_duration_s":2.940621,"scheduling_latency_s":2.940634,"redis_calls":13,"redis_duration_s":0.0015739999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00916,"completed_at":"2026-07-21T06:36:05.945Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 146.046954] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fadfd9e341f9267af3b9641a","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fadfd9e341f9267af3b9641a: start","job_status":"start","queue_duration_s":2.950726,"scheduling_latency_s":2.950739}1403gitlab # [ 146.058072] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fadfd9e341f9267af3b9641a","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fadfd9e341f9267af3b9641a: done: 0.009168 sec","job_status":"done","queue_duration_s":2.950726,"scheduling_latency_s":2.950739,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009168,"completed_at":"2026-07-21T06:36:05.955Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 146.101107] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b61b9d51a813089c63a011ba","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b61b9d51a813089c63a011ba: start","job_status":"start","queue_duration_s":2.960846,"scheduling_latency_s":2.96086}1405gitlab # [ 146.139189] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b61b9d51a813089c63a011ba","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b61b9d51a813089c63a011ba: done: 0.009178 sec","job_status":"done","queue_duration_s":2.960846,"scheduling_latency_s":2.96086,"redis_calls":13,"redis_duration_s":0.0016220000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009178,"completed_at":"2026-07-21T06:36:05.965Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 146.140246] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ff079748ce784a92cce5308","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ff079748ce784a92cce5308: start","job_status":"start","queue_duration_s":2.970964,"scheduling_latency_s":2.970976}1407gitlab # [ 146.148213] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ff079748ce784a92cce5308","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ff079748ce784a92cce5308: done: 0.00912 sec","job_status":"done","queue_duration_s":2.970964,"scheduling_latency_s":2.970976,"redis_calls":13,"redis_duration_s":0.0015370000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00912,"completed_at":"2026-07-21T06:36:05.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 146.183711] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fb396a1d81513fe003a21e7","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fb396a1d81513fe003a21e7: start","job_status":"start","queue_duration_s":2.981032,"scheduling_latency_s":2.981045}1409gitlab # [ 146.224132] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fb396a1d81513fe003a21e7","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fb396a1d81513fe003a21e7: done: 0.009223 sec","job_status":"done","queue_duration_s":2.981032,"scheduling_latency_s":2.981045,"redis_calls":13,"redis_duration_s":0.001609,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009223,"completed_at":"2026-07-21T06:36:05.985Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 146.228708] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72e872232c213f09bcb26c07","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72e872232c213f09bcb26c07: start","job_status":"start","queue_duration_s":2.991192,"scheduling_latency_s":2.991206}1411gitlab # [ 146.239683] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72e872232c213f09bcb26c07","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72e872232c213f09bcb26c07: done: 0.009113 sec","job_status":"done","queue_duration_s":2.991192,"scheduling_latency_s":2.991206,"redis_calls":13,"redis_duration_s":0.001526,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009113,"completed_at":"2026-07-21T06:36:05.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 146.280557] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:05.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b2557ba440f2ce97cf58de8","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b2557ba440f2ce97cf58de8: start","job_status":"start","queue_duration_s":3.001091,"scheduling_latency_s":3.001104}1413gitlab # [ 146.461408] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2051084926 wd_nsec: 20510849421414gitlab # [ 146.333203] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b2557ba440f2ce97cf58de8","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b2557ba440f2ce97cf58de8: done: 1.238201 sec","job_status":"done","queue_duration_s":3.001091,"scheduling_latency_s":3.001104,"redis_calls":13,"redis_duration_s":0.0017009999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000682,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":1.238201,"completed_at":"2026-07-21T06:36:07.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 146.368872] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33d519325b2ac6a9d2ba34f5","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33d519325b2ac6a9d2ba34f5: start","job_status":"start","queue_duration_s":4.275243,"scheduling_latency_s":4.27527}1416gitlab # [ 146.380018] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33d519325b2ac6a9d2ba34f5","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33d519325b2ac6a9d2ba34f5: done: 0.011211 sec","job_status":"done","queue_duration_s":4.275243,"scheduling_latency_s":4.27527,"redis_calls":13,"redis_duration_s":0.001631,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011211,"completed_at":"2026-07-21T06:36:07.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 146.380884] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97b450300b76f5e6edcce825","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97b450300b76f5e6edcce825: start","job_status":"start","queue_duration_s":4.287315,"scheduling_latency_s":4.287328}1418gitlab # [ 146.390070] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97b450300b76f5e6edcce825","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97b450300b76f5e6edcce825: done: 0.009089 sec","job_status":"done","queue_duration_s":4.287315,"scheduling_latency_s":4.287328,"redis_calls":13,"redis_duration_s":0.00161,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009089,"completed_at":"2026-07-21T06:36:07.291Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 146.435213] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ef9f7f8d0ccaa844b809981","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ef9f7f8d0ccaa844b809981: start","job_status":"start","queue_duration_s":4.297237,"scheduling_latency_s":4.29725}1420gitlab # [ 146.436237] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ef9f7f8d0ccaa844b809981","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ef9f7f8d0ccaa844b809981: done: 0.009153 sec","job_status":"done","queue_duration_s":4.297237,"scheduling_latency_s":4.29725,"redis_calls":13,"redis_duration_s":0.00155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009153,"completed_at":"2026-07-21T06:36:07.301Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 146.483171] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"614d55cfee6754f4c27b4203","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-614d55cfee6754f4c27b4203: start","job_status":"start","queue_duration_s":4.307189,"scheduling_latency_s":4.307203}1422gitlab # [ 146.484212] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"614d55cfee6754f4c27b4203","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-614d55cfee6754f4c27b4203: done: 0.009448 sec","job_status":"done","queue_duration_s":4.307189,"scheduling_latency_s":4.307203,"redis_calls":13,"redis_duration_s":0.001766,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001279,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009448,"completed_at":"2026-07-21T06:36:07.312Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 146.517158] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18f5b629c049e5355c064b03","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18f5b629c049e5355c064b03: start","job_status":"start","queue_duration_s":4.317456,"scheduling_latency_s":4.317469}1424gitlab # [ 146.564939] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18f5b629c049e5355c064b03","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18f5b629c049e5355c064b03: done: 0.008797 sec","job_status":"done","queue_duration_s":4.317456,"scheduling_latency_s":4.317469,"redis_calls":13,"redis_duration_s":0.001469,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000469,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008797,"completed_at":"2026-07-21T06:36:07.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 146.565959] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"461249ab5be2b22beecc16bd","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-461249ab5be2b22beecc16bd: start","job_status":"start","queue_duration_s":4.327036,"scheduling_latency_s":4.327049}1426gitlab # [ 146.610610] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"461249ab5be2b22beecc16bd","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-461249ab5be2b22beecc16bd: done: 0.010388 sec","job_status":"done","queue_duration_s":4.327036,"scheduling_latency_s":4.327049,"redis_calls":13,"redis_duration_s":0.0024809999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010388,"completed_at":"2026-07-21T06:36:07.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 146.611710] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bca68257edc60967b51cf49","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bca68257edc60967b51cf49: start","job_status":"start","queue_duration_s":4.338366,"scheduling_latency_s":4.338379}1428gitlab # [ 146.622109] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bca68257edc60967b51cf49","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bca68257edc60967b51cf49: done: 0.008925 sec","job_status":"done","queue_duration_s":4.338366,"scheduling_latency_s":4.338379,"redis_calls":13,"redis_duration_s":0.001693,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008925,"completed_at":"2026-07-21T06:36:07.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 146.667622] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f48978306a6e9a55c67e7fa","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f48978306a6e9a55c67e7fa: start","job_status":"start","queue_duration_s":4.34822,"scheduling_latency_s":4.348234}1430gitlab # [ 146.668283] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f48978306a6e9a55c67e7fa","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f48978306a6e9a55c67e7fa: done: 0.009797 sec","job_status":"done","queue_duration_s":4.34822,"scheduling_latency_s":4.348234,"redis_calls":13,"redis_duration_s":0.00215,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001449,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009797,"completed_at":"2026-07-21T06:36:07.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 146.708227] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03b94eadd2bd1b62366d827d","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03b94eadd2bd1b62366d827d: start","job_status":"start","queue_duration_s":4.359118,"scheduling_latency_s":4.359131}1432gitlab # [ 146.745988] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03b94eadd2bd1b62366d827d","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03b94eadd2bd1b62366d827d: done: 0.009742 sec","job_status":"done","queue_duration_s":4.359118,"scheduling_latency_s":4.359131,"redis_calls":13,"redis_duration_s":0.002227,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001493,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009742,"completed_at":"2026-07-21T06:36:07.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 146.747023] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b870f7a5cf8ced170df39ce2","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b870f7a5cf8ced170df39ce2: start","job_status":"start","queue_duration_s":4.369927,"scheduling_latency_s":4.36994}1434gitlab # [ 146.790982] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b870f7a5cf8ced170df39ce2","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b870f7a5cf8ced170df39ce2: done: 0.009809 sec","job_status":"done","queue_duration_s":4.369927,"scheduling_latency_s":4.36994,"redis_calls":13,"redis_duration_s":0.002066,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009809,"completed_at":"2026-07-21T06:36:07.375Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 146.792038] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f260c00c4548b1eaf451d067","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f260c00c4548b1eaf451d067: start","job_status":"start","queue_duration_s":4.380811,"scheduling_latency_s":4.380824}1436gitlab # [ 146.832114] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f260c00c4548b1eaf451d067","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f260c00c4548b1eaf451d067: done: 0.009964 sec","job_status":"done","queue_duration_s":4.380811,"scheduling_latency_s":4.380824,"redis_calls":13,"redis_duration_s":0.00209,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001377,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009964,"completed_at":"2026-07-21T06:36:07.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 146.834992] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d38a06c44e3b6ea5e35cf2f","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d38a06c44e3b6ea5e35cf2f: start","job_status":"start","queue_duration_s":4.391828,"scheduling_latency_s":4.391841}1438gitlab # [ 146.846122] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d38a06c44e3b6ea5e35cf2f","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d38a06c44e3b6ea5e35cf2f: done: 0.009358 sec","job_status":"done","queue_duration_s":4.391828,"scheduling_latency_s":4.391841,"redis_calls":13,"redis_duration_s":0.0017629999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009358,"completed_at":"2026-07-21T06:36:07.396Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 146.904933] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b35f76cb148f31f3aecf1e4","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b35f76cb148f31f3aecf1e4: start","job_status":"start","queue_duration_s":4.401978,"scheduling_latency_s":4.40199}1440gitlab # [ 146.905694] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b35f76cb148f31f3aecf1e4","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b35f76cb148f31f3aecf1e4: done: 0.00934 sec","job_status":"done","queue_duration_s":4.401978,"scheduling_latency_s":4.40199,"redis_calls":13,"redis_duration_s":0.00176,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001257,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00934,"completed_at":"2026-07-21T06:36:07.406Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 146.926252] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"679005986cb632cf1684bcf0","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-679005986cb632cf1684bcf0: start","job_status":"start","queue_duration_s":4.412148,"scheduling_latency_s":4.412161}1442gitlab # [ 146.969217] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"679005986cb632cf1684bcf0","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-679005986cb632cf1684bcf0: done: 0.009261 sec","job_status":"done","queue_duration_s":4.412148,"scheduling_latency_s":4.412161,"redis_calls":13,"redis_duration_s":0.001585,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009261,"completed_at":"2026-07-21T06:36:07.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 146.971128] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00598ea7d1425fbe7c45d696","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00598ea7d1425fbe7c45d696: start","job_status":"start","queue_duration_s":4.422257,"scheduling_latency_s":4.42227}1444gitlab # [ 146.982265] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00598ea7d1425fbe7c45d696","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00598ea7d1425fbe7c45d696: done: 0.00988 sec","job_status":"done","queue_duration_s":4.422257,"scheduling_latency_s":4.42227,"redis_calls":13,"redis_duration_s":0.0017590000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00988,"completed_at":"2026-07-21T06:36:07.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 147.035672] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6dfade5b08fbc75d496b797","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6dfade5b08fbc75d496b797: start","job_status":"start","queue_duration_s":4.433021,"scheduling_latency_s":4.433034}1446gitlab # [ 147.036553] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6dfade5b08fbc75d496b797","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6dfade5b08fbc75d496b797: done: 0.009205 sec","job_status":"done","queue_duration_s":4.433021,"scheduling_latency_s":4.433034,"redis_calls":13,"redis_duration_s":0.001524,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009205,"completed_at":"2026-07-21T06:36:07.437Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 147.075504] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f23f5621e4c75a58fd83502","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f23f5621e4c75a58fd83502: start","job_status":"start","queue_duration_s":4.443031,"scheduling_latency_s":4.443044}1448gitlab # [ 147.076157] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f23f5621e4c75a58fd83502","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f23f5621e4c75a58fd83502: done: 0.009054 sec","job_status":"done","queue_duration_s":4.443031,"scheduling_latency_s":4.443044,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009054,"completed_at":"2026-07-21T06:36:07.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 147.115482] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5fee66b3ec09cbfd1d57a36","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5fee66b3ec09cbfd1d57a36: start","job_status":"start","queue_duration_s":4.452882,"scheduling_latency_s":4.452895}1450gitlab # [ 147.117480] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5fee66b3ec09cbfd1d57a36","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5fee66b3ec09cbfd1d57a36: done: 0.00916 sec","job_status":"done","queue_duration_s":4.452882,"scheduling_latency_s":4.452895,"redis_calls":13,"redis_duration_s":0.0015409999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00916,"completed_at":"2026-07-21T06:36:07.457Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 147.162269] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ed72ce0686d90a646fd261c","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ed72ce0686d90a646fd261c: start","job_status":"start","queue_duration_s":4.462966,"scheduling_latency_s":4.462979}1452gitlab # [ 147.162923] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ed72ce0686d90a646fd261c","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ed72ce0686d90a646fd261c: done: 0.009005 sec","job_status":"done","queue_duration_s":4.462966,"scheduling_latency_s":4.462979,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009005,"completed_at":"2026-07-21T06:36:07.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 147.203449] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcba856bd05ded6e29cdb26a","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcba856bd05ded6e29cdb26a: start","job_status":"start","queue_duration_s":4.472999,"scheduling_latency_s":4.473013}1454gitlab # [ 147.242925] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcba856bd05ded6e29cdb26a","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcba856bd05ded6e29cdb26a: done: 0.009243 sec","job_status":"done","queue_duration_s":4.472999,"scheduling_latency_s":4.473013,"redis_calls":13,"redis_duration_s":0.0015999999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009243,"completed_at":"2026-07-21T06:36:07.477Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 147.243927] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1b863f7162f04931a4b4a6e","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1b863f7162f04931a4b4a6e: start","job_status":"start","queue_duration_s":4.483195,"scheduling_latency_s":4.483208}1456gitlab # [ 147.252462] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1b863f7162f04931a4b4a6e","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1b863f7162f04931a4b4a6e: done: 0.009217 sec","job_status":"done","queue_duration_s":4.483195,"scheduling_latency_s":4.483208,"redis_calls":13,"redis_duration_s":0.001609,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009217,"completed_at":"2026-07-21T06:36:07.487Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 147.302965] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"244a008d95110b62c071c438","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-244a008d95110b62c071c438: start","job_status":"start","queue_duration_s":4.493369,"scheduling_latency_s":4.493381}1458gitlab # [ 147.303749] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.498Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"244a008d95110b62c071c438","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-244a008d95110b62c071c438: done: 0.009405 sec","job_status":"done","queue_duration_s":4.493369,"scheduling_latency_s":4.493381,"redis_calls":13,"redis_duration_s":0.0016879999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009405,"completed_at":"2026-07-21T06:36:07.498Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 147.366696] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b002a011ae5f53f8f30c9af","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b002a011ae5f53f8f30c9af: start","job_status":"start","queue_duration_s":4.503738,"scheduling_latency_s":4.503753}1460gitlab # [ 147.367647] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b002a011ae5f53f8f30c9af","created_at":"2026-07-21T06:36:02.945Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-48b423a8b9cf45d4","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e2129230d38655af06d0dc8e8aec6425","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:02.995Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b002a011ae5f53f8f30c9af: done: 0.009167 sec","job_status":"done","queue_duration_s":4.503738,"scheduling_latency_s":4.503753,"redis_calls":13,"redis_duration_s":0.0015530000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009167,"completed_at":"2026-07-21T06:36:07.508Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 147.409958] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"a2d7eba904a1684c51300d9d","created_at":"2026-07-21T06:36:05.453Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-f1218a16f6044ebe","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"66fd2069ff6d20c65dc13aa3b4d7b207","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:05.458Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-a2d7eba904a1684c51300d9d: start","job_status":"start","queue_duration_s":2.050548,"scheduling_latency_s":2.050561}1462gitlab # [ 147.410713] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:07.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"a2d7eba904a1684c51300d9d","created_at":"2026-07-21T06:36:05.453Z","trace_propagation_headers":{"sentry-trace":"9ecf036505ec4840a6fd18128e823cc4-f1218a16f6044ebe","baggage":"sentry-trace_id=9ecf036505ec4840a6fd18128e823cc4,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"66fd2069ff6d20c65dc13aa3b4d7b207","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:05.458Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-a2d7eba904a1684c51300d9d: done: 0.02128 sec","job_status":"done","queue_duration_s":2.050548,"scheduling_latency_s":2.050561,"redis_calls":13,"redis_duration_s":0.001772,"redis_read_bytes":12,"redis_write_bytes":1665,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020254,"duration_s":0.02128,"completed_at":"2026-07-21T06:36:07.530Z","load_balancing_strategy":"primary","db_duration_s":0.004658,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 154.085162] setup-gitlab-start[2612]: W, [2026-07-21T06:36:14.986348 #2612] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1464gitlab # [ 175.930433] runuser[2611]: pam_unix(runuser:session): session closed for user gitlab1465gitlab # [ 175.934707] setup-gitlab-start[2825]: ++ jq -r .token1466gitlab # [ 175.935227] setup-gitlab-start[2826]: +++ date -d '+7 days' +%F1467gitlab # [ 175.938369] setup-gitlab-start[2824]: ++ curl -fs -X POST -H 'Authorization: Bearer ecd80fa5e3d13bff2e760e97cb1e5952f31281b541ad4ba41ceab74b80b81163' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-28"}' http://gitlab/api/v4/users/1/personal_access_tokens1468gitlab # [ 176.447251] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:37.347Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d5f78eef887572eefb36ecb4","created_at":"2026-07-21T06:36:37.325Z","trace_propagation_headers":{"sentry-trace":"5688c08fff8f44fa8f89e872a00c1c88-5723b5d57115441a","baggage":"sentry-trace_id=5688c08fff8f44fa8f89e872a00c1c88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P803YQEFWWR5WR1KZS7EJ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:6b33680886cd244e55aedb7b65a04a27354cf521aee3ba49f7e070d87898537d","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:37.344Z","job_size_bytes":451,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d5f78eef887572eefb36ecb4: start","job_status":"start","queue_duration_s":0.003076,"scheduling_latency_s":0.003127}1469gitlab # [ 176.450576] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:37 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 4881470gitlab # [ 176.488628] setup-gitlab-start[2575]: + TOKEN=glpat-9Pol8hCM9QDTGlZzH5q7IW86MQp1OjEH.01.0w0rfl7rj1471gitlab # [ 176.489366] setup-gitlab-start[2575]: + test -n glpat-9Pol8hCM9QDTGlZzH5q7IW86MQp1OjEH.01.0w0rfl7rj1472gitlab # [ 176.490479] setup-gitlab-start[2575]: + test glpat-9Pol8hCM9QDTGlZzH5q7IW86MQp1OjEH.01.0w0rfl7rj '!=' null1473gitlab # [ 176.492082] setup-gitlab-start[2575]: + mkdir -p /var/lib/secrets1474gitlab # [ 176.493310] setup-gitlab-start[2575]: + echo glpat-9Pol8hCM9QDTGlZzH5q7IW86MQp1OjEH.01.0w0rfl7rj1475gitlab # [ 176.494441] setup-gitlab-start[2575]: + chmod 644 /var/lib/secrets/gitlab-token1476gitlab # [ 176.504632] setup-gitlab-start[2575]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-9Pol8hCM9QDTGlZzH5q7IW86MQp1OjEH.01.0w0rfl7rj' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1477gitlab # [ 176.627092] sidekiq-cluster[2058]: {"severity":"WARN","time":"2026-07-21T06:36:37.528Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d5f78eef887572eefb36ecb4","created_at":"2026-07-21T06:36:37.325Z","trace_propagation_headers":{"sentry-trace":"5688c08fff8f44fa8f89e872a00c1c88-5723b5d57115441a","baggage":"sentry-trace_id=5688c08fff8f44fa8f89e872a00c1c88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P803YQEFWWR5WR1KZS7EJ","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:6b33680886cd244e55aedb7b65a04a27354cf521aee3ba49f7e070d87898537d","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:37.344Z","job_size_bytes":451,"pid":2058,"sidekiq_tid":"lypm","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d5f78eef887572eefb36ecb4: fail: 0.179947 sec","job_status":"fail","queue_duration_s":0.003076,"scheduling_latency_s":0.003127,"redis_calls":11,"redis_duration_s":0.016557,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.013762,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002047,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000748,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000317918,"duration_s":0.179947,"completed_at":"2026-07-21T06:36:37.527Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.042165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 177.322788] gitaly[1354]: time="2026-07-21T06:36:38.223Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.223" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1479gitlab # [ 177.323321] gitaly[1354]: time="2026-07-21T06:36:38.224Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.223" grpc.time_ms=0.617 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1480gitlab # [ 177.439128] gitaly[1354]: time="2026-07-21T06:36:38.340Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.120" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.340" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1481gitlab # [ 177.439611] gitaly[1354]: time="2026-07-21T06:36:38.341Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.120" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.340" grpc.time_ms=0.35 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1482gitlab # [ 177.480212] gitaly[1354]: time="2026-07-21T06:36:38.381Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 diskcache=72cfe8c7-1c95-435f-a0f9-3c6c5731ee0f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-21T06:36:48.125" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.345" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1483gitlab # [ 177.482159] gitaly[1354]: time="2026-07-21T06:36:38.382Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=32 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-21T06:36:48.125" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.345" grpc.time_ms=36.453 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1484gitlab # [ 177.494406] gitaly[1354]: time="2026-07-21T06:36:38.396Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.175" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.395" pid=1354 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1485gitlab # [ 177.496136] gitaly[1354]: time="2026-07-21T06:36:38.396Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.175" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.395" grpc.time_ms=0.43 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1486gitlab # [ 177.508359] gitaly[1354]: time="2026-07-21T06:36:38.410Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T06:36:48.179" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.400" grpc.time_ms=9.433 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1487gitlab # [ 177.540800] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:38.441Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b47947218c23bd620825ddd6","created_at":"2026-07-21T06:36:38.417Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.439Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b47947218c23bd620825ddd6: start","job_status":"start","queue_duration_s":0.00256,"scheduling_latency_s":0.002611}1488gitlab # [ 177.550484] gitaly[1354]: time="2026-07-21T06:36:38.452Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.231" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.452" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1489gitlab # [ 177.551629] gitaly[1354]: time="2026-07-21T06:36:38.452Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.231" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.452" grpc.time_ms=0.369 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1490gitlab # [ 177.589537] gitaly[1354]: time="2026-07-21T06:36:38.491Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 diskcache=77d96bbf-fd6a-4984-bcd3-71971003323a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-21T06:36:48.235" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.455" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1491gitlab # [ 177.589849] gitaly[1354]: time="2026-07-21T06:36:38.491Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=32 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-21T06:36:48.235" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.455" grpc.time_ms=35.668 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1492gitlab # [ 177.594363] gitaly[1354]: time="2026-07-21T06:36:38.496Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.275" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.495" pid=1354 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1493gitlab # [ 177.595468] gitaly[1354]: time="2026-07-21T06:36:38.496Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.275" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.495" grpc.time_ms=0.372 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1494gitlab # [ 177.602522] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:38.503Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b47947218c23bd620825ddd6","created_at":"2026-07-21T06:36:38.417Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.439Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b47947218c23bd620825ddd6: done: 0.062088 sec","job_status":"done","queue_duration_s":0.00256,"scheduling_latency_s":0.002611,"redis_calls":10,"redis_duration_s":0.0025180000000000003,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001493,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.036376969,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.062088,"completed_at":"2026-07-21T06:36:38.503Z","load_balancing_strategy":"primary","db_duration_s":0.010804,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1495gitlab # [ 177.656729] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:38.505Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"943a04c094b786d0b59fe42c","created_at":"2026-07-21T06:36:38.444Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.448Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-943a04c094b786d0b59fe42c: start","job_status":"start","queue_duration_s":0.056775,"scheduling_latency_s":0.056799}1496gitlab # [ 177.657522] gitaly[1354]: time="2026-07-21T06:36:38.507Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T06:36:48.281" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.501" grpc.time_ms=5.537 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1497gitlab # [ 177.658193] gitaly[1354]: time="2026-07-21T06:36:38.527Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:48.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.527" grpc.time_ms=0.104 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1498gitlab # [ 177.666088] gitaly[1354]: time="2026-07-21T06:36:38.531Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-21T06:36:48.010" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.531" grpc.time_ms=0.129 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1499gitlab # [ 177.707847] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:38.585Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"943a04c094b786d0b59fe42c","created_at":"2026-07-21T06:36:38.444Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.448Z","job_size_bytes":2,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-943a04c094b786d0b59fe42c: done: 0.080259 sec","job_status":"done","queue_duration_s":0.056775,"scheduling_latency_s":0.056799,"redis_calls":14,"redis_duration_s":0.002254,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000467,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001092,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000695,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.061459767,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.080259,"completed_at":"2026-07-21T06:36:38.585Z","load_balancing_strategy":"primary","db_duration_s":0.016526,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1500gitlab # [ 177.754520] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:38.656Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c46ba84b42286c8ee6b8f165","created_at":"2026-07-21T06:36:38.649Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.655Z","job_size_bytes":5,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c46ba84b42286c8ee6b8f165: start","job_status":"start","queue_duration_s":0.001057,"scheduling_latency_s":0.001087}1501gitlab # [ 177.797323] gitaly[1354]: time="2026-07-21T06:36:38.698Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P80NHPXFYRTYNZ3BCBXX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T06:36:48.172" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:38.692" grpc.time_ms=6.142 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1502gitlab # [ 177.830564] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:38.732Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c46ba84b42286c8ee6b8f165","created_at":"2026-07-21T06:36:38.649Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.655Z","job_size_bytes":5,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c46ba84b42286c8ee6b8f165: done: 0.076048 sec","job_status":"done","queue_duration_s":0.001057,"scheduling_latency_s":0.001087,"redis_calls":12,"redis_duration_s":0.002219,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001424,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000323,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000336914,"duration_s":0.076048,"completed_at":"2026-07-21T06:36:38.732Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022258,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1503gitlab # [ 177.871658] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:38.773Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"3020817ac836e3f5c8e95f4d","created_at":"2026-07-21T06:36:38.673Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.678Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-3020817ac836e3f5c8e95f4d: start","job_status":"start","queue_duration_s":0.094843,"scheduling_latency_s":0.094865}1504gitlab # [ 178.059894] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:38.961Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"3020817ac836e3f5c8e95f4d","created_at":"2026-07-21T06:36:38.673Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.678Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-3020817ac836e3f5c8e95f4d: done: 0.188 sec","job_status":"done","queue_duration_s":0.094843,"scheduling_latency_s":0.094865,"redis_calls":11,"redis_duration_s":0.002602,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000764,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001426,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.152,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.152,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.130227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000305346,"duration_s":0.188,"completed_at":"2026-07-21T06:36:38.961Z","load_balancing_strategy":"primary","db_duration_s":0.059188,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 178.137700] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:39.038Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"3315ae44e320b6333f3d6ec2","created_at":"2026-07-21T06:36:38.680Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.683Z","job_size_bytes":89,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-3315ae44e320b6333f3d6ec2: start","job_status":"start","queue_duration_s":0.355444,"scheduling_latency_s":0.355489}1506gitlab # [ 178.176812] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:39 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 16501507gitlab # [ 178.178830] setup-gitlab-start[2575]: + mkdir -p /tmp/test-flake1508gitlab # [ 178.181448] setup-gitlab-start[2575]: + cd /tmp/test-flake1509gitlab # [ 178.182087] setup-gitlab-start[2575]: + git init -b master1510gitlab # [ 178.205849] setup-gitlab-start[2869]: Initialized empty Git repository in /tmp/test-flake/.git/1511gitlab # [ 178.208261] setup-gitlab-start[2575]: + git config user.name test1512gitlab # [ 178.210317] setup-gitlab-start[2575]: + git config user.email test@example.com1513gitlab # [ 178.215789] setup-gitlab-start[2575]: + cat1514gitlab # [ 178.220141] setup-gitlab-start[2575]: + git add flake.nix1515gitlab # [ 178.223178] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:39.124Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"3315ae44e320b6333f3d6ec2","created_at":"2026-07-21T06:36:38.680Z","trace_propagation_headers":{"sentry-trace":"d4b95084bf40410ba2ddfe54e2ab424e-f3815048ff594264","baggage":"sentry-trace_id=d4b95084bf40410ba2ddfe54e2ab424e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P80NHPXFYRTYNZ3BCBXX0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:38.683Z","job_size_bytes":89,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-3315ae44e320b6333f3d6ec2: done: 0.086204 sec","job_status":"done","queue_duration_s":0.355444,"scheduling_latency_s":0.355489,"redis_calls":11,"redis_duration_s":0.002483,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000684,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001361,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025953,"duration_s":0.086204,"completed_at":"2026-07-21T06:36:39.124Z","load_balancing_strategy":"primary","db_duration_s":0.027167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516gitlab # [ 178.250226] setup-gitlab-start[2575]: + git commit -m 'initial commit'1517gitlab # [ 178.251116] setup-gitlab-start[2874]: [master (root-commit) 81bca38] initial commit1518gitlab # [ 178.251650] setup-gitlab-start[2874]: 1 file changed, 10 insertions(+)1519gitlab # [ 178.252162] setup-gitlab-start[2874]: create mode 100644 flake.nix1520gitlab # [ 178.252646] setup-gitlab-start[2575]: + git remote add origin http://oauth2:glpat-9Pol8hCM9QDTGlZzH5q7IW86MQp1OjEH.01.0w0rfl7rj@gitlab/root/test-flake.git1521gitlab # [ 178.253137] setup-gitlab-start[2575]: + git push -u origin master1522gitlab # [ 178.349804] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 801523gitlab # [ 178.539748] gitaly[1354]: time="2026-07-21T06:36:39.440Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82ENXYPABQ1TKV0ZWGRF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:39.428" grpc.time_ms=11.777 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1524gitlab # [ 178.561719] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1881525gitlab # [ 178.745776] gitaly[1354]: time="2026-07-21T06:36:39.647Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T06:36:49.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:39.636" grpc.time_ms=10.825 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1526gitlab # [ 178.811262] gitaly[1354]: time="2026-07-21T06:36:39.713Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-21T06:37:09.002" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-21T06:36:39.702" grpc.time_ms=10.478 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1527gitlab # [ 178.869766] gitaly[1354]: time="2026-07-21T06:36:39.771Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KY1P82MTV28B58XX59VGJR2H duration_ms=194ns method=POST pid=1354 status=200 url="http://unix/api/v4/internal/allowed"1528gitlab # [ 178.918970] gitaly[1354]: time="2026-07-21T06:36:39.820Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KY1P82MTV28B58XX59VGJR2H duration_ms=48ns method=POST pid=1354 status=200 url="http://unix/api/v4/internal/pre_receive"1529gitlab # [ 178.929457] gitaly[1354]: time="2026-07-21T06:36:39.821Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5321 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:39.575" grpc.time_ms=245.837 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1530gitlab # [ 178.988543] gitaly[1354]: time="2026-07-21T06:36:39.890Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:39.889" grpc.time_ms=0.912 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1531gitlab # [ 179.054977] gitaly[1354]: time="2026-07-21T06:36:39.956Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:39.955" grpc.time_ms=1.327 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1532gitlab # [ 179.106344] gitaly[1354]: time="2026-07-21T06:36:40.008Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:40.007" grpc.time_ms=0.627 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1533gitlab # [ 179.188680] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:40.089Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 81bca3848bad4535c1e676bce30116356170a9b3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a77fccb2a599a1e1900f1b68","created_at":"2026-07-21T06:36:40.083Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-56cce718b1604ffb","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.087Z","job_size_bytes":185,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a77fccb2a599a1e1900f1b68: start","job_status":"start","queue_duration_s":0.001835,"scheduling_latency_s":0.001883}1534gitlab # [ 179.222261] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:40.118Z","message":"changes: \"0000000000000000000000000000000000000000 81bca3848bad4535c1e676bce30116356170a9b3 refs/heads/master\\n\""}1535gitlab # [ 179.228398] gitaly[1354]: time="2026-07-21T06:36:40.130Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T06:36:50.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:40.124" grpc.time_ms=6.18 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1536gitlab # [ 179.249396] gitaly[1354]: time="2026-07-21T06:36:40.150Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T06:36:50.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:40.144" grpc.time_ms=6.543 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1537gitlab # [ 179.262583] gitaly[1354]: time="2026-07-21T06:36:40.160Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1567 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-21T06:36:50.023" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:40.144" grpc.time_ms=16.657 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1538gitlab # [ 179.276062] gitaly[1354]: time="2026-07-21T06:36:40.169Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KY1P82MTV28B58XX59VGJR2H duration_ms=116ns method=POST pid=1354 status=200 url="http://unix/api/v4/internal/post_receive"1539gitlab # [ 179.279066] gitaly[1354]: time="2026-07-21T06:36:40.169Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4815 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:40.052" grpc.time_ms=117.154 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1540gitlab # [ 179.289139] gitaly[1354]: time="2026-07-21T06:36:40.175Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:50.026" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:40.166" grpc.time_ms=9.387 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1541gitlab # [ 179.302479] gitaly[1354]: time="2026-07-21T06:36:40.201Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H diskcache=57e114fd-f13a-435f-a142-bf19734698c1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:39.501" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1542gitlab # [ 179.311466] setup-gitlab-start[2878]: To http://gitlab/root/test-flake.git1543gitlab # [ 179.312397] setup-gitlab-start[2878]: * [new branch] master -> master1544gitlab # [ 179.313338] setup-gitlab-start[2878]: branch 'master' set up to track 'origin/master'.1545gitlab # [ 179.314647] gitlab-workhorse[1292]: time="2026-07-21T06:36:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521546gitlab # [ 179.316469] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:40 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 7511547gitlab: (finished: waiting for unit setup-gitlab.service, in 180.03 seconds)1548gitlab # [ 179.318312] gitaly[1354]: time="2026-07-21T06:36:40.201Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=249 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13399 command.oublock=32 command.real_time_ms=688 command.spawn_token_fork_ms=0 command.system_time_ms=163 command.user_time_ms=86 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:39.501" grpc.time_ms=699.751 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1549gitlab: waiting for unit nixbot.service1550gitlab # [ 179.330471] systemd[1]: Finished setup-gitlab.service.1551gitlab # [ 179.330831] gitaly[1354]: time="2026-07-21T06:36:40.210Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-21T06:36:50.061" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:40.201" grpc.time_ms=9.23 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1552gitlab # [ 179.331476] systemd[1]: Starting nixbot CI service...1553gitlab # [ 179.349974] gitaly[1354]: time="2026-07-21T06:36:40.251Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3941 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:40.251" grpc.time_ms=0.294 pid=1354 span.kind=server system=grpc1554gitlab # [ 179.384177] gitaly[1354]: time="2026-07-21T06:36:40.286Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3943 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:40.285" grpc.time_ms=0.299 pid=1354 span.kind=server system=grpc1555gitlab # [ 179.402408] gitaly[1354]: time="2026-07-21T06:36:40.304Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H diskcache=7ee5ad19-5c58-4b7b-9da0-aa0e043e5ba2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T06:36:50.079" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:40.220" pid=1354 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1556gitlab # [ 179.404040] gitaly[1354]: time="2026-07-21T06:36:40.304Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=44 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4760 command.oublock=8 command.real_time_ms=75 command.spawn_token_fork_ms=1 command.system_time_ms=31 command.user_time_ms=13 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T06:36:50.079" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:40.220" grpc.time_ms=84.114 pid=1354 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1557gitlab # [ 179.424089] gitaly[1354]: time="2026-07-21T06:36:40.318Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T06:36:50.178" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:40.318" grpc.time_ms=0.105 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1558gitlab # [ 179.428324] gitaly[1354]: time="2026-07-21T06:36:40.330Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=934 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-21T06:36:50.181" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:40.321" grpc.time_ms=8.962 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1559gitlab # [ 179.481331] gitaly[1354]: time="2026-07-21T06:36:40.383Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-21T06:37:10.079" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:40.379" grpc.time_ms=3.559 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1560gitlab # [ 179.500979] gitaly[1354]: time="2026-07-21T06:36:40.402Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-21T06:36:50.262" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:40.402" grpc.time_ms=0.152 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1561gitlab # [ 179.515198] gitaly[1354]: time="2026-07-21T06:36:40.417Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-21T06:36:50.273" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-21T06:36:40.413" grpc.time_ms=3.209 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1562gitlab # [ 179.544894] gitaly[1354]: time="2026-07-21T06:36:40.446Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:50.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:40.441" grpc.time_ms=5.413 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1563gitlab # [ 179.552143] gitaly[1354]: time="2026-07-21T06:36:40.454Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:50.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:40.449" grpc.time_ms=4.765 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1564gitlab # [ 179.566441] gitaly[1354]: time="2026-07-21T06:36:40.468Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-21T06:37:10.161" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:40.461" grpc.time_ms=7.091 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1565gitlab # [ 179.844574] gitaly[1354]: time="2026-07-21T06:36:40.746Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-21T06:36:50.003" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:40.743" grpc.time_ms=2.168 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1566gitlab # [ 179.878337] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:40.779Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 81bca3848bad4535c1e676bce30116356170a9b3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a77fccb2a599a1e1900f1b68","created_at":"2026-07-21T06:36:40.083Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-56cce718b1604ffb","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.087Z","job_size_bytes":185,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a77fccb2a599a1e1900f1b68: done: 0.690606 sec","job_status":"done","queue_duration_s":0.001835,"scheduling_latency_s":0.001883,"gitaly_calls":13,"gitaly_duration_s":0.204589,"redis_calls":196,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.040355,"redis_read_bytes":1445,"redis_write_bytes":19018,"redis_cache_calls":14,"redis_cache_duration_s":0.002969,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.007812,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.000921,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00453,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.013103,"redis_repository_cache_read_bytes":233,"redis_repository_cache_write_bytes":6319,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.01102,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.181,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.194,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.403397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001658311,"exclusive_lock_hold_duration_s":0.024847013,"duration_s":0.690606,"completed_at":"2026-07-21T06:36:40.779Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052485,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1567gitlab # [ 179.928044] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:40.829Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"5389cf92bf85a80e933f3c34","created_at":"2026-07-21T06:36:40.180Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-983b820f93a742a8","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.184Z","job_size_bytes":46,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-5389cf92bf85a80e933f3c34: start","job_status":"start","queue_duration_s":0.645128,"scheduling_latency_s":0.64517}1568gitlab # [ 179.991322] gitaly[1354]: time="2026-07-21T06:36:40.893Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-21T12:36:40.890" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:40.891" grpc.time_ms=1.989 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1569gitlab # [ 180.009806] gitaly[1354]: time="2026-07-21T06:36:40.911Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-21T06:37:10.598" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:40.899" grpc.time_ms=12.094 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1570gitlab # [ 180.030595] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:40.932Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"5389cf92bf85a80e933f3c34","created_at":"2026-07-21T06:36:40.180Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-983b820f93a742a8","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.184Z","job_size_bytes":46,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-5389cf92bf85a80e933f3c34: done: 0.102689 sec","job_status":"done","queue_duration_s":0.645128,"scheduling_latency_s":0.64517,"gitaly_calls":2,"gitaly_duration_s":0.018122,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007752,"redis_read_bytes":68,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001611,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000538,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001797,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003097,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000305626,"duration_s":0.102689,"completed_at":"2026-07-21T06:36:40.931Z","load_balancing_strategy":"primary","db_duration_s":0.014087,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1571gitlab # [ 180.077463] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:40.979Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"356d26a4ec69549c0910cb60","created_at":"2026-07-21T06:36:40.194Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-983b820f93a742a8","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.197Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-356d26a4ec69549c0910cb60: start","job_status":"start","queue_duration_s":0.781249,"scheduling_latency_s":0.781274}1572gitlab # [ 180.123936] gitaly[1354]: time="2026-07-21T06:36:41.025Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H diskcache=32ddae6b-1c33-4ed2-9397-58888adc3482 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-21T12:36:40.997" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:40.998" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1573gitlab # [ 180.124932] gitaly[1354]: time="2026-07-21T06:36:41.026Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1286 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-21T12:36:40.997" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:40.998" grpc.time_ms=27.787 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1574gitlab # [ 180.153429] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:41.054Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"356d26a4ec69549c0910cb60","created_at":"2026-07-21T06:36:40.194Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-983b820f93a742a8","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.197Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-356d26a4ec69549c0910cb60: done: 0.075765 sec","job_status":"done","queue_duration_s":0.781249,"scheduling_latency_s":0.781274,"gitaly_calls":1,"gitaly_duration_s":0.030088,"redis_calls":15,"redis_duration_s":0.003724,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000675,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001205,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.057365110999999996,"duration_s":0.075765,"completed_at":"2026-07-21T06:36:41.054Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575gitlab # [ 180.202368] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:41.103Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"745bc025419d6823d77e363a","created_at":"2026-07-21T06:36:40.314Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-983b820f93a742a8","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.316Z","job_size_bytes":23,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-745bc025419d6823d77e363a: start","job_status":"start","queue_duration_s":0.786199,"scheduling_latency_s":0.786249}1576gitlab # [ 180.247656] gitaly[1354]: time="2026-07-21T06:36:41.149Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-21T06:37:11.040" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:41.140" grpc.time_ms=8.874 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1577gitlab # [ 180.254772] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:41.156Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"745bc025419d6823d77e363a","created_at":"2026-07-21T06:36:40.314Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-983b820f93a742a8","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.316Z","job_size_bytes":23,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-745bc025419d6823d77e363a: done: 0.05366 sec","job_status":"done","queue_duration_s":0.786199,"scheduling_latency_s":0.786249,"gitaly_calls":1,"gitaly_duration_s":0.011444,"redis_calls":16,"redis_duration_s":0.004706,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000285,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002133,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05366,"completed_at":"2026-07-21T06:36:41.156Z","load_balancing_strategy":"primary","db_duration_s":0.004353,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1578gitlab # [ 180.260164] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:41.157Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f1440e803804019bc83a5236","created_at":"2026-07-21T06:36:40.698Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-983b820f93a742a8","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:274d988c6377b6a424de1d67f3c4a05ff355dff5e801d745392b9686b6582bc4","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.705Z","job_size_bytes":467,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f1440e803804019bc83a5236: start","job_status":"start","queue_duration_s":0.452228,"scheduling_latency_s":0.452246}1579gitlab # [ 180.285699] gitaly[1354]: time="2026-07-21T06:36:41.187Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:11.086" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:41.186" grpc.time_ms=0.566 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1580gitlab # [ 180.334857] gitaly[1354]: time="2026-07-21T06:36:41.236Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=46 command.spawn_token_fork_ms=42 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-21T06:36:51.008" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:41.189" grpc.time_ms=47.439 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1581gitlab # [ 180.450456] gitaly[1354]: time="2026-07-21T06:36:41.352Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T06:37:11.250" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:41.350" grpc.time_ms=1.167 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1582gitlab # [ 180.454270] gitaly[1354]: time="2026-07-21T06:36:41.356Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=42 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-21T06:37:11.253" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:41.354" grpc.time_ms=1.711 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 180.504764] nixbot[2981]: {"ts": "2026-07-21T06:36:41.406Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1584gitlab # [ 180.526582] nixbot[2981]: {"ts": "2026-07-21T06:36:41.428Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1585gitlab # [ 180.532766] gitaly[1354]: time="2026-07-21T06:36:41.434Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:51.243" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:41.424" grpc.time_ms=10.313 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1586gitlab # [ 180.540169] nixbot[2981]: {"ts": "2026-07-21T06:36:41.442Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1587gitlab # [ 180.543918] nixbot[2981]: {"ts": "2026-07-21T06:36:41.445Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1588gitlab # [ 180.547038] nixbot[2981]: {"ts": "2026-07-21T06:36:41.448Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1589gitlab # [ 180.551313] nixbot[2981]: {"ts": "2026-07-21T06:36:41.453Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1590gitlab # [ 180.554273] nixbot[2981]: {"ts": "2026-07-21T06:36:41.456Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1591gitlab # [ 180.557869] nixbot[2981]: {"ts": "2026-07-21T06:36:41.459Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1592gitlab # [ 180.566593] nixbot[2981]: {"ts": "2026-07-21T06:36:41.468Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1593gitlab # [ 180.570678] nixbot[2981]: {"ts": "2026-07-21T06:36:41.472Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1594gitlab # [ 180.575260] nixbot[2981]: {"ts": "2026-07-21T06:36:41.477Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1595gitlab # [ 180.582288] nixbot[2981]: {"ts": "2026-07-21T06:36:41.484Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1596gitlab # [ 180.587458] nixbot[2981]: {"ts": "2026-07-21T06:36:41.489Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1597gitlab # [ 180.590139] nixbot[2981]: {"ts": "2026-07-21T06:36:41.492Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1598gitlab # [ 180.595604] nixbot[2981]: {"ts": "2026-07-21T06:36:41.497Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1599gitlab # [ 180.599731] nixbot[2981]: {"ts": "2026-07-21T06:36:41.501Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1600gitlab # [ 180.603422] nixbot[2981]: {"ts": "2026-07-21T06:36:41.505Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1601gitlab # [ 180.607456] nixbot[2981]: {"ts": "2026-07-21T06:36:41.509Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1602gitlab # [ 180.610884] nixbot[2981]: {"ts": "2026-07-21T06:36:41.512Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1603gitlab # [ 180.613177] nixbot[2981]: {"ts": "2026-07-21T06:36:41.514Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1604gitlab # [ 180.617060] nixbot[2981]: {"ts": "2026-07-21T06:36:41.518Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1605gitlab # [ 180.620896] nixbot[2981]: {"ts": "2026-07-21T06:36:41.522Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1606gitlab # [ 180.623121] nixbot[2981]: {"ts": "2026-07-21T06:36:41.524Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "effect_deps"}1607gitlab # [ 180.916396] nixbot[2981]: INFO: Started server process [2981]1608gitlab # [ 180.916836] nixbot[2981]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1609gitlab # [ 180.920918] nixbot[2981]: INFO: ::ffff:127.0.0.1:40160 - "GET /health HTTP/1.1" 200 OK1610gitlab # [ 180.925757] systemd[1]: Started nixbot CI service.1611gitlab # [ 180.926916] systemd[1]: Reached target Multi-User System.1612gitlab # [ 180.929123] systemd[1]: Startup finished in 1.226s (kernel) + 3.891s (initrd) + 2min 55.810s (userspace) = 3min 929ms.1613gitlab # [ 180.990907] gitaly[1354]: time="2026-07-21T06:36:41.889Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-21T06:37:11.779" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:41.879" grpc.time_ms=9.225 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1614gitlab # [ 181.030052] gitaly[1354]: time="2026-07-21T06:36:41.931Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=36 command.spawn_token_fork_ms=28 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P82MTV28B58XX59VGJR2H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-21T06:37:11.791" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:41.892" grpc.time_ms=39.625 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1615gitlab # [ 181.072357] sidekiq-cluster[2058]: {"severity":"WARN","time":"2026-07-21T06:36:41.974Z","class":"CreatePipelineWorker","correlation_id":"01KY1P82MTV28B58XX59VGJR2H","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"81bca3848bad4535c1e676bce30116356170a9b3","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"81bca3848bad4535c1e676bce30116356170a9b3","ref":"refs/heads/master"},"retry":0}1616gitlab # [ 181.085432] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:41.980Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f1440e803804019bc83a5236","created_at":"2026-07-21T06:36:40.698Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-983b820f93a742a8","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:274d988c6377b6a424de1d67f3c4a05ff355dff5e801d745392b9686b6582bc4","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.705Z","job_size_bytes":467,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f1440e803804019bc83a5236: done: 0.823457 sec","job_status":"done","queue_duration_s":0.452228,"scheduling_latency_s":0.452246,"gitaly_calls":7,"gitaly_duration_s":0.124706,"redis_calls":53,"redis_duration_s":0.023614,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.014033,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004695,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002483,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.608795,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000436648,"duration_s":0.823457,"completed_at":"2026-07-21T06:36:41.980Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057112,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1617gitlab # [ 181.090931] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:41.982Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6a2ecc947e60ab6f53d417ca","created_at":"2026-07-21T06:36:40.920Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-e2bcf3c72cea43f9","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.924Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6a2ecc947e60ab6f53d417ca: start","job_status":"start","queue_duration_s":1.058152,"scheduling_latency_s":1.058167}1618gitlab # [ 181.135081] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:42.010Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6a2ecc947e60ab6f53d417ca","created_at":"2026-07-21T06:36:40.920Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-e2bcf3c72cea43f9","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:40.924Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6a2ecc947e60ab6f53d417ca: done: 0.028479 sec","job_status":"done","queue_duration_s":1.058152,"scheduling_latency_s":1.058167,"redis_calls":19,"redis_duration_s":0.0029389999999999998,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000568,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000351,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001468,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0045489020000000005,"duration_s":0.028479,"completed_at":"2026-07-21T06:36:42.010Z","load_balancing_strategy":"primary","db_duration_s":0.005054,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1619gitlab # [ 181.145996] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:42.011Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"662e181b3a1264340b377534","created_at":"2026-07-21T06:36:42.002Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-a5d1b670cfd1468d","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:42.006Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-662e181b3a1264340b377534: start","job_status":"start","queue_duration_s":0.005764,"scheduling_latency_s":0.005778}1620gitlab # [ 181.190291] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:42.091Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"662e181b3a1264340b377534","created_at":"2026-07-21T06:36:42.002Z","trace_propagation_headers":{"sentry-trace":"8f40e312600e4572a6113b82237306b1-a5d1b670cfd1468d","baggage":"sentry-trace_id=8f40e312600e4572a6113b82237306b1,sentry-release=Unknown"},"correlation_id":"01KY1P82MTV28B58XX59VGJR2H","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:42.006Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-662e181b3a1264340b377534: done: 0.080136 sec","job_status":"done","queue_duration_s":0.005764,"scheduling_latency_s":0.005778,"redis_calls":11,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001071,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000196114,"duration_s":0.080136,"completed_at":"2026-07-21T06:36:42.091Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1621gitlab # [ 181.402702] gitaly[1354]: time="2026-07-21T06:36:42.304Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P84Z236YEHNFC36VK3C9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-21T06:37:12.088" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:42.288" grpc.time_ms=15.525 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1622gitlab: (finished: waiting for unit nixbot.service, in 2.11 seconds)1623gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1624gitlab # [ 181.548591] nixbot[2981]: INFO: ::ffff:127.0.0.1:40162 - "GET /health HTTP/1.1" 200 OK1625gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1626subtest: project discovered and webhook registered1627gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1628gitlab # [ 181.663230] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 761629gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.12 seconds)1630gitlab # [ 181.731210] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:42 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 8061631gitlab # [ 181.733296] nixbot[2981]: {"ts": "2026-07-21T06:36:42.635Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1632gitlab # [ 181.748687] nixbot[2981]: {"ts": "2026-07-21T06:36:42.650Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1633gitlab # [ 181.826118] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 621634gitlab # [ 181.827678] nixbot[2981]: {"ts": "2026-07-21T06:36:42.729Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1635gitlab # [ 181.829993] nixbot[2981]: {"ts": "2026-07-21T06:36:42.731Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1636gitlab # [ 181.981527] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:42 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1401637gitlab # [ 181.987489] nixbot[2981]: {"ts": "2026-07-21T06:36:42.889Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1638gitlab # [ 182.103176] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1061639gitlab # [ 182.103729] nixbot[2981]: {"ts": "2026-07-21T06:36:43.005Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1640gitlab # [ 182.172242] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 601641gitlab # [ 182.177212] nixbot[2981]: {"ts": "2026-07-21T06:36:43.075Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1642gitlab # [ 182.250971] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 721643gitlab # [ 182.256988] nixbot[2981]: {"ts": "2026-07-21T06:36:43.158Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1644gitlab # [ 182.329786] gitaly[1354]: time="2026-07-21T06:36:43.230Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P868XMQ6T0X3GMZS97M8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-21T06:37:13.018" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:43.218" grpc.time_ms=12.115 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1645gitlab # [ 182.363476] gitaly[1354]: time="2026-07-21T06:36:43.265Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P868XMQ6T0X3GMZS97M8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:13.038" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:43.239" grpc.time_ms=25.946 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1646gitlab # [ 182.379345] gitaly[1354]: time="2026-07-21T06:36:43.281Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P868XMQ6T0X3GMZS97M8Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-21T06:36:53.049" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:43.270" grpc.time_ms=10.893 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1647gitlab # [ 182.464884] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 2001648gitlab # [ 182.466174] nixbot[2981]: {"ts": "2026-07-21T06:36:43.367Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1649gitlab # [ 182.594256] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1231650gitlab # [ 182.596094] nixbot[2981]: {"ts": "2026-07-21T06:36:43.497Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1651gitlab # [ 182.600915] nixbot[2981]: {"ts": "2026-07-21T06:36:43.502Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1652gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1653gitlab # [ 182.721114] gitaly[1354]: time="2026-07-21T06:36:43.622Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86MP9KB79ACHDYMBNMDX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.607" grpc.time_ms=14.981 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1654gitlab # [ 182.748562] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 831655gitlab # [ 182.766763] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 561656gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds)1657(finished: subtest: project discovered and webhook registered, in 1.22 seconds)1658subtest: push delivers the webhook1659gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1660gitlab # [ 182.812072] gitaly[1354]: time="2026-07-21T06:36:43.713Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86QDEBYA0HQNMYW3Z70R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.701" pid=1354 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1661gitlab # [ 182.822784] gitaly[1354]: time="2026-07-21T06:36:43.713Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86QDEBYA0HQNMYW3Z70R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.701" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1662gitlab # [ 182.836709] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 201663gitlab # [ 182.841030] gitaly[1354]: time="2026-07-21T06:36:43.714Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86QDEBYA0HQNMYW3Z70R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.701" grpc.time_ms=12.498 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1664gitlab # [ 182.856337] gitlab-workhorse[1292]: time="2026-07-21T06:36:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491665gitlab # [ 182.858404] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1121666gitlab # [ 182.901091] gitaly[1354]: time="2026-07-21T06:36:43.802Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86TG8JY9H6NEFVQASGWH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.794" grpc.time_ms=8.047 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1667gitlab # [ 182.914531] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 741668gitlab # [ 182.916992] gitaly[1354]: time="2026-07-21T06:36:43.819Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86V3Z33VA91807A8YPQW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.793" pid=1354 remote_ip="2001:db8:1::1" request_sha=b7955b94794f51640171260e076ed33265ce3d3f response_bytes=228 span.kind=server system=grpc1669gitlab # Enumerating objects: 5, done.1670gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1671gitlab # Delta compression using up to 4 threads1672gitlab # [ 182.929548] gitaly[1354]: time="2026-07-21T06:36:43.819Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1203 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86V3Z33VA91807A8YPQW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.793" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1673gitlab # [ 182.929966] gitlab-workhorse[1292]: time="2026-07-21T06:36:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2281674gitlab # Compressing objects: 50% (1/2)[ 182.930435] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.54.0" 721675gitlab # Compressing objects: 100% (2/2)[ 182.931446] gitaly[1354]: time="2026-07-21T06:36:43.819Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1203 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86V3Z33VA91807A8YPQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.793" grpc.time_ms=25.932 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1676gitlab # Compressing objects: 100% (2/2), done.1677gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done.1678gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1679gitlab # [ 183.031779] gitaly[1354]: time="2026-07-21T06:36:43.933Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86ZHTMCMBSY76NBKKCZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.926" grpc.time_ms=7.124 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1680gitlab # [ 183.044578] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:43 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 441681gitlab # [ 183.103083] gitaly[1354]: time="2026-07-21T06:36:44.004Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87106GMD72RXM5T09HH4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.993" pid=1354 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1682gitlab # [ 183.128267] gitaly[1354]: time="2026-07-21T06:36:44.004Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87106GMD72RXM5T09HH4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.993" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1683gitlab # [ 183.128745] gitlab-workhorse[1292]: time="2026-07-21T06:36:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671684gitlab # [ 183.129227] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 681685gitlab # [ 183.129785] gitaly[1354]: time="2026-07-21T06:36:44.004Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87106GMD72RXM5T09HH4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.993" grpc.time_ms=10.325 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1686gitlab # [ 183.130426] gitaly[1354]: time="2026-07-21T06:36:44.014Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-21T06:36:53.008" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:43.999" grpc.time_ms=14.871 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1687gitlab # [ 183.195218] gitaly[1354]: time="2026-07-21T06:36:44.097Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-21T06:37:14.090" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-21T06:36:44.090" grpc.time_ms=6.185 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688gitlab # [ 183.207362] gitaly[1354]: time="2026-07-21T06:36:44.109Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 duration_ms=162ns method=POST pid=1354 status=200 url="http://unix/api/v4/internal/allowed"1689gitlab # [ 183.232916] gitaly[1354]: time="2026-07-21T06:36:44.134Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P875C2RA220DMJ16DXQ3V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:44.128" grpc.time_ms=6.206 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1690gitlab # [ 183.233592] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 581691gitlab # [ 183.235464] gitaly[1354]: time="2026-07-21T06:36:44.136Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 duration_ms=27ns method=POST pid=1354 status=200 url="http://unix/api/v4/internal/pre_receive"1692gitlab # [ 183.235580] gitaly[1354]: time="2026-07-21T06:36:44.137Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5321 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:43.946" grpc.time_ms=190.339 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1693gitlab # [ 183.279079] gitaly[1354]: time="2026-07-21T06:36:44.181Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:44.180" grpc.time_ms=0.421 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1694gitlab # [ 183.320117] gitaly[1354]: time="2026-07-21T06:36:44.220Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:44.219" grpc.time_ms=0.481 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1695gitlab # [ 183.328352] gitaly[1354]: time="2026-07-21T06:36:44.230Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87835AD61VMK9Q21NGXP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:44.216" pid=1354 remote_ip="2001:db8:1::1" request_sha=69dcef142412daaffeeda28f6446e517599d516e response_bytes=214 span.kind=server system=grpc1696gitlab # [ 183.329145] gitaly[1354]: time="2026-07-21T06:36:44.230Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1157 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87835AD61VMK9Q21NGXP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:44.216" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1697gitlab # [ 183.330175] gitlab-workhorse[1292]: time="2026-07-21T06:36:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2141698gitlab # [ 183.330652] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 214 "" "git/2.54.0" 681699gitlab # [ 183.332496] gitaly[1354]: time="2026-07-21T06:36:44.231Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1157 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87835AD61VMK9Q21NGXP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:44.216" grpc.time_ms=13.885 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1700gitlab # [ 183.393029] gitaly[1354]: time="2026-07-21T06:36:44.294Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:44.294" grpc.time_ms=0.333 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1701gitlab # [ 183.436968] gitaly[1354]: time="2026-07-21T06:36:44.338Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:14.338" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:44.338" grpc.time_ms=0.446 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1702gitlab # [ 183.453811] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:44.354Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","81bca3848bad4535c1e676bce30116356170a9b3 ee886a6781a18511736d0485295ed219b36bdc47 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"63803032894f14518928eb5f","created_at":"2026-07-21T06:36:44.350Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-b69fd4e0d4644ff2","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.353Z","job_size_bytes":185,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-63803032894f14518928eb5f: start","job_status":"start","queue_duration_s":0.001853,"scheduling_latency_s":0.001885}1703gitlab # [ 183.465251] gitaly[1354]: time="2026-07-21T06:36:44.367Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 duration_ms=40ns method=POST pid=1354 status=200 url="http://unix/api/v4/internal/post_receive"1704gitlab # [ 183.465968] gitaly[1354]: time="2026-07-21T06:36:44.367Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4815 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:44.325" grpc.time_ms=42.038 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1705gitlab # [ 183.470736] gitaly[1354]: time="2026-07-21T06:36:44.372Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=5 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-21T06:37:14.364" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:44.364" grpc.time_ms=8.363 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1706gitlab # [ 183.471657] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:44.373Z","message":"changes: \"81bca3848bad4535c1e676bce30116356170a9b3 ee886a6781a18511736d0485295ed219b36bdc47 refs/heads/master\\n\""}1707gitlab # [ 183.490335] gitaly[1354]: time="2026-07-21T06:36:44.392Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:54.206" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:44.386" grpc.time_ms=5.154 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1708gitlab # [ 183.497792] gitaly[1354]: time="2026-07-21T06:36:44.399Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T06:36:54.215" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:44.395" grpc.time_ms=4.36 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1709gitlab # [ 183.530033] gitaly[1354]: time="2026-07-21T06:36:44.430Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=15 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-21T06:36:54.229" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:44.409" grpc.time_ms=20.748 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1710gitlab # [ 183.547590] gitaly[1354]: time="2026-07-21T06:36:44.448Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=656 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=8 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-21T06:37:14.334" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:44.435" grpc.time_ms=13.675 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1711gitlab # [ 183.563658] gitaly[1354]: time="2026-07-21T06:36:44.465Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=6 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:54.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:44.439" grpc.time_ms=25.653 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1712gitlab # [ 183.566909] gitaly[1354]: time="2026-07-21T06:36:44.468Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-21T06:36:54.285" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-21T06:36:44.465" grpc.time_ms=3.12 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1713gitlab # [ 183.590703] gitaly[1354]: time="2026-07-21T06:36:44.492Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 diskcache=c58115e4-e45c-4f08-b983-685214e0dec5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.886" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1714gitlab # [ 183.591385] gitlab-workhorse[1292]: time="2026-07-21T06:36:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521715gitlab # [ 183.592204] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:44 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 6541716gitlab # To http://gitlab/root/test-flake.git1717gitlab # 81bca38..ee886a6 master -> master1718gitlab # [ 183.604089] gitaly[1354]: time="2026-07-21T06:36:44.492Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=180 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13285 command.oublock=32 command.real_time_ms=541 command.spawn_token_fork_ms=1 command.system_time_ms=112 command.user_time_ms=68 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:43.886" grpc.time_ms=605.82 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1719gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.85 seconds)1720gitlab: must succeed: git -C /tmp/test-flake rev-parse master1721gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1722gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee886a6781a18511736d0485295ed219b36bdc47'1723gitlab # [ 183.631956] gitaly[1354]: time="2026-07-21T06:36:44.533Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:54.086" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:44.526" grpc.time_ms=6.339 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1724gitlab # [ 183.656640] gitaly[1354]: time="2026-07-21T06:36:44.558Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-21T06:36:54.128" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:44.558" grpc.time_ms=0.169 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1725gitlab # [ 183.673575] nixbot[2981]: INFO: ::ffff:127.0.0.1:35186 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee886a6781a18511736d0485295ed219b36bdc47 HTTP/1.1" 200 OK1726gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee886a6781a18511736d0485295ed219b36bdc47', in 0.04 seconds)1727gitlab # [ 183.678502] gitaly[1354]: time="2026-07-21T06:36:44.580Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=619 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-21T06:36:54.142" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:44.573" grpc.time_ms=6.419 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1728gitlab # [ 183.707083] gitaly[1354]: time="2026-07-21T06:36:44.608Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:44.608" grpc.time_ms=0.265 pid=1354 span.kind=server system=grpc1729gitlab # [ 183.733683] gitaly[1354]: time="2026-07-21T06:36:44.635Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4160 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:44.635" grpc.time_ms=0.24 pid=1354 span.kind=server system=grpc1730gitlab # [ 183.750686] gitaly[1354]: time="2026-07-21T06:36:44.652Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ diskcache=c65bdde5-cac8-4779-80cf-6d0189802d75 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T06:36:54.151" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:44.581" pid=1354 ref=refs/keep-around/81bca3848bad4535c1e676bce30116356170a9b3 remote_ip="2001:db8:1::1" revision=81bca3848bad4535c1e676bce30116356170a9b3 span.kind=server system=grpc username=root1731gitlab # [ 183.761138] gitaly[1354]: time="2026-07-21T06:36:44.663Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=63 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5254 command.oublock=8 command.real_time_ms=62 command.spawn_token_fork_ms=0 command.system_time_ms=44 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T06:36:54.151" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:44.581" grpc.time_ms=70.882 pid=1354 ref=refs/keep-around/81bca3848bad4535c1e676bce30116356170a9b3 remote_ip="2001:db8:1::1" revision=81bca3848bad4535c1e676bce30116356170a9b3 span.kind=server system=grpc username=root1732gitlab # [ 183.770150] gitaly[1354]: time="2026-07-21T06:36:44.672Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-21T06:36:54.230" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:44.671" grpc.time_ms=0.989 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1733gitlab # [ 183.782656] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:44.684Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","81bca3848bad4535c1e676bce30116356170a9b3 ee886a6781a18511736d0485295ed219b36bdc47 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"63803032894f14518928eb5f","created_at":"2026-07-21T06:36:44.350Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-b69fd4e0d4644ff2","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.353Z","job_size_bytes":185,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-63803032894f14518928eb5f: done: 0.329313 sec","job_status":"done","queue_duration_s":0.001853,"scheduling_latency_s":0.001885,"gitaly_calls":7,"gitaly_duration_s":0.126894,"redis_calls":108,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.027218,"redis_read_bytes":349,"redis_write_bytes":12822,"redis_cache_calls":5,"redis_cache_duration_s":0.001228,"redis_cache_read_bytes":13,"redis_cache_write_bytes":347,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0003,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.00075,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6768,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.005184,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006894,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.012862,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.023,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.147003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000484139,"exclusive_lock_hold_duration_s":0.011672712,"duration_s":0.329313,"completed_at":"2026-07-21T06:36:44.684Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03183,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1734gitlab # [ 183.792101] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:44.685Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"eb85e75d77cf7d1a17f8660a","created_at":"2026-07-21T06:36:44.401Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-d3eb263213ad484e","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.403Z","job_size_bytes":46,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-eb85e75d77cf7d1a17f8660a: start","job_status":"start","queue_duration_s":0.281992,"scheduling_latency_s":0.28201}1735gitlab # [ 183.806766] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:44.697Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"eb85e75d77cf7d1a17f8660a","created_at":"2026-07-21T06:36:44.401Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-d3eb263213ad484e","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.403Z","job_size_bytes":46,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-eb85e75d77cf7d1a17f8660a: done: 0.011868 sec","job_status":"done","queue_duration_s":0.281992,"scheduling_latency_s":0.28201,"redis_calls":9,"redis_duration_s":0.0013770000000000002,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000134,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000612,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011868,"completed_at":"2026-07-21T06:36:44.697Z","load_balancing_strategy":"primary","db_duration_s":0.001,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1736gitlab # [ 183.862042] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:44.698Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"dbed52f0efcde545ef0f60d1","created_at":"2026-07-21T06:36:44.406Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-d3eb263213ad484e","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.407Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-dbed52f0efcde545ef0f60d1: start","job_status":"start","queue_duration_s":0.290613,"scheduling_latency_s":0.290626}1737gitlab # [ 183.862827] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:44.744Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"dbed52f0efcde545ef0f60d1","created_at":"2026-07-21T06:36:44.406Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-d3eb263213ad484e","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.407Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-dbed52f0efcde545ef0f60d1: done: 0.046263 sec","job_status":"done","queue_duration_s":0.290613,"scheduling_latency_s":0.290626,"gitaly_calls":1,"gitaly_duration_s":0.026527,"redis_calls":12,"redis_duration_s":0.001683,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000688,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000356,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000639,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.035330189,"duration_s":0.046263,"completed_at":"2026-07-21T06:36:44.744Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1738gitlab # [ 183.932495] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:44.745Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"18fecdfec29fef18c0e0abe8","created_at":"2026-07-21T06:36:44.579Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-d3eb263213ad484e","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b6afb31dd7de6ef2d442d006a96f5c3567615d373f25062dbf2bcf60651ac239","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.581Z","job_size_bytes":467,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-18fecdfec29fef18c0e0abe8: start","job_status":"start","queue_duration_s":0.164076,"scheduling_latency_s":0.16409}1739gitlab # [ 183.933232] gitaly[1354]: time="2026-07-21T06:36:44.733Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 diskcache=c8a827af-9503-45ab-8cf5-96209ef22276 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-21T12:36:44.707" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:44.708" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1740gitlab # [ 183.933872] gitaly[1354]: time="2026-07-21T06:36:44.733Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1645 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-21T12:36:44.707" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:44.708" grpc.time_ms=25.101 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741gitlab # [ 183.939760] gitaly[1354]: time="2026-07-21T06:36:44.752Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:14.751" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:44.752" grpc.time_ms=0.242 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1742gitlab # [ 183.980730] gitaly[1354]: time="2026-07-21T06:36:44.760Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:14.659" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:44.759" grpc.time_ms=0.358 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1743gitlab # [ 183.981546] nixbot[2981]: {"ts": "2026-07-21T06:36:44.875Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/81bca3848bad4535c1e676bce30116356170a9b3 \"HTTP/1.1 201 Created\""}1744gitlab # [ 183.982085] nixbot[2981]: {"ts": "2026-07-21T06:36:44.881Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-81bca3848bad-5d7fd626", "argv0": "bwrap"}1745gitlab # [ 183.982763] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:44 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/81bca3848bad4535c1e676bce30116356170a9b3 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 5811746gitlab # [ 183.983499] gitaly[1354]: time="2026-07-21T06:36:44.767Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-21T06:36:54.022" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:44.762" grpc.time_ms=5 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1747gitlab # [ 183.984148] gitaly[1354]: time="2026-07-21T06:36:44.797Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T06:37:14.697" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:44.797" grpc.time_ms=0.576 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1748gitlab # [ 183.991538] gitaly[1354]: time="2026-07-21T06:36:44.804Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=63 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-21T06:37:14.700" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:44.801" grpc.time_ms=3.098 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1749gitlab # [ 184.005927] gitaly[1354]: time="2026-07-21T06:36:44.839Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:54.092" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:44.832" grpc.time_ms=7.245 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1750gitlab # [ 184.118182] systemd[1]: Started Nix Daemon.1751gitlab # [ 184.162434] nix-daemon[3278]: accepted connection from pid 3276, user nixbot1752gitlab # [ 184.167212] nix-daemon[3278]: accepted connection from pid 3277, user nixbot1753gitlab # [ 184.326743] gitaly[1354]: time="2026-07-21T06:36:45.228Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=968 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-21T06:37:15.110" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:45.211" grpc.time_ms=17.076 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754gitlab # [ 184.371934] gitaly[1354]: time="2026-07-21T06:36:45.273Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P86XYJJHZ2YHMQQFS1G59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-21T06:37:15.161" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:45.261" grpc.time_ms=11.829 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755gitlab # [ 184.387851] sidekiq-cluster[2058]: {"severity":"WARN","time":"2026-07-21T06:36:45.289Z","class":"CreatePipelineWorker","correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"81bca3848bad4535c1e676bce30116356170a9b3","after":"ee886a6781a18511736d0485295ed219b36bdc47","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"ee886a6781a18511736d0485295ed219b36bdc47","ref":"refs/heads/master"},"retry":0}1756gitlab # [ 184.395368] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.296Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"18fecdfec29fef18c0e0abe8","created_at":"2026-07-21T06:36:44.579Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-d3eb263213ad484e","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b6afb31dd7de6ef2d442d006a96f5c3567615d373f25062dbf2bcf60651ac239","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.581Z","job_size_bytes":467,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-18fecdfec29fef18c0e0abe8: done: 0.551279 sec","job_status":"done","queue_duration_s":0.164076,"scheduling_latency_s":0.16409,"gitaly_calls":7,"gitaly_duration_s":0.089328,"redis_calls":33,"redis_duration_s":0.007864,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001152,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001355,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004191,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001166,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.076,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.073,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.375962,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.551279,"completed_at":"2026-07-21T06:36:45.296Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075825,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1757gitlab # [ 184.436376] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.299Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"7a0670ea-be9a-432c-9bf7-52bf1636c516\"}"],"class":"WebHookWorker","jid":"c1d11ad17c62a1f20066c412","created_at":"2026-07-21T06:36:44.659Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-d3eb263213ad484e","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d577d6d18cfff23414bb8f3e1c21d30c34f0717f556f54eab56e6bbef1a714fe","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.664Z","job_size_bytes":1749,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c1d11ad17c62a1f20066c412: start","job_status":"start","queue_duration_s":0.635013,"scheduling_latency_s":0.635047}1758gitlab # [ 184.523306] nixbot[2981]: INFO: ::ffff:127.0.0.1:35196 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1759gitlab # [ 184.536386] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.437Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"7a0670ea-be9a-432c-9bf7-52bf1636c516\"}"],"class":"WebHookWorker","jid":"c1d11ad17c62a1f20066c412","created_at":"2026-07-21T06:36:44.659Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-d3eb263213ad484e","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d577d6d18cfff23414bb8f3e1c21d30c34f0717f556f54eab56e6bbef1a714fe","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.664Z","job_size_bytes":1749,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c1d11ad17c62a1f20066c412: done: 0.138184 sec","job_status":"done","queue_duration_s":0.635013,"scheduling_latency_s":0.635047,"redis_calls":19,"redis_duration_s":0.004259,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001081,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000325,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001799,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.010096814000007726,"cpu_s":0.112753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000333003,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.138184,"completed_at":"2026-07-21T06:36:45.437Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1760gitlab # [ 184.544720] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.438Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"147652fd4469339c32d0dfaa","created_at":"2026-07-21T06:36:44.757Z","trace_propagation_headers":{"sentry-trace":"9b240a064f6b4fec806e35e2a8db01c8-45aa846fc03a4209","baggage":"sentry-trace_id=9b240a064f6b4fec806e35e2a8db01c8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P87C4XYASHP8X4J9N4ZFJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.762Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-147652fd4469339c32d0dfaa: start","job_status":"start","queue_duration_s":0.676243,"scheduling_latency_s":0.676259}1761gitlab # [ 184.600226] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.478Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"147652fd4469339c32d0dfaa","created_at":"2026-07-21T06:36:44.757Z","trace_propagation_headers":{"sentry-trace":"9b240a064f6b4fec806e35e2a8db01c8-45aa846fc03a4209","baggage":"sentry-trace_id=9b240a064f6b4fec806e35e2a8db01c8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P87C4XYASHP8X4J9N4ZFJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.762Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-147652fd4469339c32d0dfaa: done: 0.039458 sec","job_status":"done","queue_duration_s":0.676243,"scheduling_latency_s":0.676259,"redis_calls":18,"redis_duration_s":0.004186,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002454,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000300877,"duration_s":0.039458,"completed_at":"2026-07-21T06:36:45.478Z","load_balancing_strategy":"primary","db_duration_s":0.012849,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1762gitlab # [ 184.612671] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.479Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"745a8ec29693afbad6644cf3","created_at":"2026-07-21T06:36:44.762Z","trace_propagation_headers":{"sentry-trace":"9b240a064f6b4fec806e35e2a8db01c8-45aa846fc03a4209","baggage":"sentry-trace_id=9b240a064f6b4fec806e35e2a8db01c8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P87C4XYASHP8X4J9N4ZFJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.769Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-745a8ec29693afbad6644cf3: start","job_status":"start","queue_duration_s":0.7095,"scheduling_latency_s":0.709514}1763gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee886a6781a18511736d0485295ed219b36bdc47'1764gitlab # [ 184.634103] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.535Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"745a8ec29693afbad6644cf3","created_at":"2026-07-21T06:36:44.762Z","trace_propagation_headers":{"sentry-trace":"9b240a064f6b4fec806e35e2a8db01c8-45aa846fc03a4209","baggage":"sentry-trace_id=9b240a064f6b4fec806e35e2a8db01c8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P87C4XYASHP8X4J9N4ZFJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.769Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-745a8ec29693afbad6644cf3: done: 0.056012 sec","job_status":"done","queue_duration_s":0.7095,"scheduling_latency_s":0.709514,"redis_calls":15,"redis_duration_s":0.0035830000000000002,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000609,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001431,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000423518,"duration_s":0.056012,"completed_at":"2026-07-21T06:36:45.535Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015094,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1765gitlab # [ 184.676578] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.537Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0a4474cc9fd27f6dc9aa4160","created_at":"2026-07-21T06:36:44.860Z","trace_propagation_headers":{"sentry-trace":"9b240a064f6b4fec806e35e2a8db01c8-45aa846fc03a4209","baggage":"sentry-trace_id=9b240a064f6b4fec806e35e2a8db01c8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P87C4XYASHP8X4J9N4ZFJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.864Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0a4474cc9fd27f6dc9aa4160: start","job_status":"start","queue_duration_s":0.672714,"scheduling_latency_s":0.672745}1766gitlab # [ 184.744565] nixbot[2981]: INFO: ::ffff:127.0.0.1:35198 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee886a6781a18511736d0485295ed219b36bdc47 HTTP/1.1" 200 OK1767gitlab # [ 184.745063] gitaly[1354]: time="2026-07-21T06:36:45.645Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T06:37:15.544" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:45.645" grpc.time_ms=0.797 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1768gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee886a6781a18511736d0485295ed219b36bdc47', in 0.08 seconds)1769gitlab # [ 184.785076] gitaly[1354]: time="2026-07-21T06:36:45.687Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:15.585" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:45.686" grpc.time_ms=0.579 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1770gitlab # [ 184.815089] gitaly[1354]: time="2026-07-21T06:36:45.716Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P88P6KSKSCHM87QG6YCBQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:45.708" grpc.time_ms=7.561 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1771gitlab # [ 184.815844] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:45 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 791772gitlab # [ 184.830976] gitaly[1354]: time="2026-07-21T06:36:45.732Z" level=info msg="finished unary call with code OK" catfile.duration_ms=15 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=15 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P87C4XYASHP8X4J9N4ZFJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:15.617" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:45.717" grpc.time_ms=15.399 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773gitlab # [ 184.843759] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.744Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0a4474cc9fd27f6dc9aa4160","created_at":"2026-07-21T06:36:44.860Z","trace_propagation_headers":{"sentry-trace":"9b240a064f6b4fec806e35e2a8db01c8-45aa846fc03a4209","baggage":"sentry-trace_id=9b240a064f6b4fec806e35e2a8db01c8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P87C4XYASHP8X4J9N4ZFJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:44.864Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0a4474cc9fd27f6dc9aa4160: done: 0.206885 sec","job_status":"done","queue_duration_s":0.672714,"scheduling_latency_s":0.672745,"gitaly_calls":3,"gitaly_duration_s":0.02197,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.053258,"redis_read_bytes":208,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.022967,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001178,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001202,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000193,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003239,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000384,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.00014,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092707,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.174058156,"duration_s":0.206885,"completed_at":"2026-07-21T06:36:45.744Z","load_balancing_strategy":"primary","db_duration_s":0.059412,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1774gitlab # [ 184.895368] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.745Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c1d11ad17c62a1f20066c412"],"class":"WebHooks::LogExecutionWorker","jid":"16a07b6f5b1c06c54ea5a99b","created_at":"2026-07-21T06:36:45.427Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-10bc843d424c4256","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d9964aea9e672079c2d1cb923680de215bc8300e9d072fde79a7f1a6d2a72da3","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:45.432Z","job_size_bytes":2507,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-16a07b6f5b1c06c54ea5a99b: start","job_status":"start","queue_duration_s":0.313428,"scheduling_latency_s":0.313443}1775gitlab # [ 184.911853] gitaly[1354]: time="2026-07-21T06:36:45.759Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P88RP7Y0C28SVQS1WS0V5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:45.751" pid=1354 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1776gitlab # [ 184.921544] gitlab-workhorse[1292]: time="2026-07-21T06:36:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671777gitlab # [ 184.923321] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:45 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 411778gitlab # [ 184.925403] gitlab-workhorse[1292]: time="2026-07-21T06:36:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3411779gitlab # [ 184.927369] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:45 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 341 "" "git/2.54.0" 571780gitlab # [ 184.927521] gitaly[1354]: time="2026-07-21T06:36:45.759Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P88RP7Y0C28SVQS1WS0V5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:45.751" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1781gitlab # [ 184.928291] gitaly[1354]: time="2026-07-21T06:36:45.760Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P88RP7Y0C28SVQS1WS0V5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:45.751" grpc.time_ms=8.327 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1782gitlab # [ 184.928399] gitaly[1354]: time="2026-07-21T06:36:45.818Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P88T2HKBPAA9XCRG0CVA0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:45.803" pid=1354 remote_ip="2001:db8:1::1" request_sha=4c070922e73a95a73b7b0d253e110133bc663ae5 response_bytes=341 span.kind=server system=grpc1783gitlab # [ 184.929123] gitaly[1354]: time="2026-07-21T06:36:45.818Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1187 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P88T2HKBPAA9XCRG0CVA0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:45.803" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1784gitlab # [ 184.968610] gitaly[1354]: time="2026-07-21T06:36:45.819Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1187 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P88T2HKBPAA9XCRG0CVA0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:45.803" grpc.time_ms=15.544 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1785gitlab # [ 184.969236] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.865Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c1d11ad17c62a1f20066c412"],"class":"WebHooks::LogExecutionWorker","jid":"16a07b6f5b1c06c54ea5a99b","created_at":"2026-07-21T06:36:45.427Z","trace_propagation_headers":{"sentry-trace":"46786b7be19b4f8486b10282a302f35a-10bc843d424c4256","baggage":"sentry-trace_id=46786b7be19b4f8486b10282a302f35a,sentry-release=Unknown"},"correlation_id":"01KY1P86XYJJHZ2YHMQQFS1G59","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d9964aea9e672079c2d1cb923680de215bc8300e9d072fde79a7f1a6d2a72da3","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:45.432Z","job_size_bytes":2507,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-16a07b6f5b1c06c54ea5a99b: done: 0.120303 sec","job_status":"done","queue_duration_s":0.313428,"scheduling_latency_s":0.313443,"redis_calls":13,"redis_duration_s":0.005626,"redis_read_bytes":139,"redis_write_bytes":1387,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.000794,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":641,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000337,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.024,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.024,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207568,"duration_s":0.120303,"completed_at":"2026-07-21T06:36:45.865Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016484,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1786gitlab # [ 184.988751] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.866Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"dbfd9436d37d8fbdf8291ee1","created_at":"2026-07-21T06:36:45.626Z","trace_propagation_headers":{"sentry-trace":"9b240a064f6b4fec806e35e2a8db01c8-e4f1b4e5d7a64c12","baggage":"sentry-trace_id=9b240a064f6b4fec806e35e2a8db01c8,sentry-release=Unknown"},"correlation_id":"01KY1P87C4XYASHP8X4J9N4ZFJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:45.634Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-dbfd9436d37d8fbdf8291ee1: start","job_status":"start","queue_duration_s":0.23232,"scheduling_latency_s":0.232334}1787gitlab # [ 185.003205] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:45.894Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"dbfd9436d37d8fbdf8291ee1","created_at":"2026-07-21T06:36:45.626Z","trace_propagation_headers":{"sentry-trace":"9b240a064f6b4fec806e35e2a8db01c8-e4f1b4e5d7a64c12","baggage":"sentry-trace_id=9b240a064f6b4fec806e35e2a8db01c8,sentry-release=Unknown"},"correlation_id":"01KY1P87C4XYASHP8X4J9N4ZFJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:45.634Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-dbfd9436d37d8fbdf8291ee1: done: 0.027482 sec","job_status":"done","queue_duration_s":0.23232,"scheduling_latency_s":0.232334,"redis_calls":19,"redis_duration_s":0.0024400000000000003,"redis_read_bytes":653,"redis_write_bytes":1843,"redis_feature_flag_calls":11,"redis_feature_flag_duration_s":0.001391,"redis_feature_flag_read_bytes":646,"redis_feature_flag_write_bytes":1151,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197791,"duration_s":0.027482,"completed_at":"2026-07-21T06:36:45.894Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1788gitlab # [ 185.149426] gitaly[1354]: time="2026-07-21T06:36:46.051Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KY1P891WM7M0YXMHXH0JGXVW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:46.042" grpc.time_ms=8.668 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1789gitlab # [ 185.167084] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:46 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 561790gitlab # [ 185.215942] gitaly[1354]: time="2026-07-21T06:36:46.117Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P893Q6235JHBRXF6VPHBY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:46.104" pid=1354 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=216 span.kind=server system=grpc1791gitlab # [ 185.216734] gitaly[1354]: time="2026-07-21T06:36:46.117Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P893Q6235JHBRXF6VPHBY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:46.104" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1792gitlab # [ 185.220205] gitaly[1354]: time="2026-07-21T06:36:46.120Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P893Q6235JHBRXF6VPHBY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:46.104" grpc.time_ms=12.824 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1793gitlab # [ 185.222077] gitlab-workhorse[1292]: time="2026-07-21T06:36:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161794gitlab # [ 185.223381] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.54.0" 491795gitlab # [ 185.297245] nixbot[2981]: {"ts": "2026-07-21T06:36:46.199Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1796gitlab # [ 185.302690] nix-daemon[3285]: unexpected Nix daemon error: error: interrupted by the user1797gitlab # [ 185.351746] gitaly[1354]: time="2026-07-21T06:36:46.253Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:16.251" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:46.252" grpc.time_ms=0.867 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798gitlab # [ 185.356504] gitaly[1354]: time="2026-07-21T06:36:46.258Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:16.057" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:46.257" grpc.time_ms=0.564 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799gitlab # [ 185.382601] gitaly[1354]: time="2026-07-21T06:36:46.284Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-21T06:37:16.077" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:46.278" grpc.time_ms=6.051 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800gitlab # [ 185.396313] gitaly[1354]: time="2026-07-21T06:36:46.298Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:56.073" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:46.293" grpc.time_ms=4.617 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab # [ 185.433131] gitaly[1354]: time="2026-07-21T06:36:46.335Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-21T06:36:56.114" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:46.334" grpc.time_ms=0.181 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1802gitlab # [ 185.448331] gitaly[1354]: time="2026-07-21T06:36:46.350Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:56.224" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:46.344" grpc.time_ms=5.231 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803gitlab # [ 185.460701] gitaly[1354]: time="2026-07-21T06:36:46.362Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-21T06:36:56.125" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:46.345" grpc.time_ms=16.892 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804gitlab # [ 185.486709] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.387Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"516bebee233fadba4525e506","created_at":"2026-07-21T06:36:46.384Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-1922bd1359e048a9","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.386Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-516bebee233fadba4525e506: start","job_status":"start","queue_duration_s":0.001272,"scheduling_latency_s":0.001301}1805gitlab # [ 185.494403] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/81bca3848bad4535c1e676bce30116356170a9b3 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 1781806gitlab # [ 185.495301] nixbot[2981]: {"ts": "2026-07-21T06:36:46.397Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/81bca3848bad4535c1e676bce30116356170a9b3 \"HTTP/1.1 201 Created\""}1807gitlab # [ 185.510533] gitaly[1354]: time="2026-07-21T06:36:46.412Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:46.412" grpc.time_ms=0.273 pid=1354 span.kind=server system=grpc1808gitlab # [ 185.538935] gitaly[1354]: time="2026-07-21T06:36:46.440Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4160 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:46.439" grpc.time_ms=0.881 pid=1354 span.kind=server system=grpc1809gitlab # [ 185.558189] gitaly[1354]: time="2026-07-21T06:36:46.460Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 diskcache=c52990e8-f495-4acf-81ab-5b7b01263d9f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T06:36:56.143" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:46.364" pid=1354 ref=refs/keep-around/ee886a6781a18511736d0485295ed219b36bdc47 remote_ip="2001:db8:1::1" revision=ee886a6781a18511736d0485295ed219b36bdc47 span.kind=server system=grpc username=root1810gitlab # [ 185.568581] gitaly[1354]: time="2026-07-21T06:36:46.460Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=41 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5409 command.oublock=8 command.real_time_ms=86 command.spawn_token_fork_ms=0 command.system_time_ms=24 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T06:36:56.143" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T06:36:46.364" grpc.time_ms=96.189 pid=1354 ref=refs/keep-around/ee886a6781a18511736d0485295ed219b36bdc47 remote_ip="2001:db8:1::1" revision=ee886a6781a18511736d0485295ed219b36bdc47 span.kind=server system=grpc username=root1811gitlab # [ 185.568908] gitaly[1354]: time="2026-07-21T06:36:46.463Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:16.262" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:46.462" grpc.time_ms=0.395 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812gitlab # [ 185.569922] gitaly[1354]: time="2026-07-21T06:36:46.471Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P89E352HMDH0M2JTBSPHA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:16.262" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:46.462" grpc.time_ms=8.302 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab # [ 185.619921] gitaly[1354]: time="2026-07-21T06:36:46.521Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:46.521" grpc.time_ms=0.273 pid=1354 span.kind=server system=grpc1814gitlab # [ 185.644276] gitaly[1354]: time="2026-07-21T06:36:46.546Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:46.545" grpc.time_ms=0.33 pid=1354 span.kind=server system=grpc1815gitlab # [ 185.670380] gitaly[1354]: time="2026-07-21T06:36:46.572Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:46.572" grpc.time_ms=0.242 pid=1354 span.kind=server system=grpc1816gitlab # [ 185.688837] gitaly[1354]: time="2026-07-21T06:36:46.590Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 diskcache=d4036618-21fe-4b00-b20e-8a825c7b4b51 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:16.357" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:46.457" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab # [ 185.697751] gitaly[1354]: time="2026-07-21T06:36:46.590Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=91 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7281 command.oublock=0 command.real_time_ms=124 command.spawn_token_fork_ms=0 command.system_time_ms=48 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:16.357" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:46.457" grpc.time_ms=133.144 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818gitlab # [ 185.742812] gitaly[1354]: time="2026-07-21T06:36:46.644Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T06:37:16.543" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:46.644" grpc.time_ms=0.557 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1819gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee886a6781a18511736d0485295ed219b36bdc47'1820gitlab # [ 185.754849] gitaly[1354]: time="2026-07-21T06:36:46.656Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P89E352HMDH0M2JTBSPHA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:36:56.130" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:46.651" grpc.time_ms=4.896 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 185.767203] gitaly[1354]: time="2026-07-21T06:36:46.668Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:16.567" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:46.667" grpc.time_ms=0.463 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1822gitlab # [ 185.793351] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee886a6781a18511736d0485295ed219b36bdc47 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 4931823gitlab # [ 185.796208] nixbot[2981]: {"ts": "2026-07-21T06:36:46.695Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee886a6781a18511736d0485295ed219b36bdc47 \"HTTP/1.1 201 Created\""}1824gitlab # [ 185.799161] gitaly[1354]: time="2026-07-21T06:36:46.700Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:16.599" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:46.699" grpc.time_ms=0.448 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1825gitlab # [ 185.810486] nixbot[2981]: INFO: ::ffff:127.0.0.1:35212 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee886a6781a18511736d0485295ed219b36bdc47 HTTP/1.1" 200 OK1826gitlab # [ 185.812431] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/81bca3848bad4535c1e676bce30116356170a9b3 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 3071827gitlab # [ 185.815191] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.709Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3e14b09841a79104108c5a8c","created_at":"2026-07-21T06:36:46.708Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"516bebee233fadba4525e506","job_size_bytes":3,"pid":2058,"job_status":"deduplicated","message":"PipelineProcessWorker JID-3e14b09841a79104108c5a8c: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1828gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee886a6781a18511736d0485295ed219b36bdc47', in 0.07 seconds)1829(finished: subtest: push delivers the webhook, in 3.06 seconds)1830subtest: eval and build post commit statuses1831gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1832gitlab # [ 185.829243] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.710Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"516bebee233fadba4525e506","created_at":"2026-07-21T06:36:46.384Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-1922bd1359e048a9","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.386Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2058,"job_status":"rescheduled","message":"PipelineProcessWorker JID-516bebee233fadba4525e506: rescheduled"}1833gitlab # [ 185.842854] nixbot[2981]: {"ts": "2026-07-21T06:36:46.714Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-ee886a6781a1-f40a30ab", "argv0": "bwrap"}1834gitlab # [ 185.843208] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.716Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"516bebee233fadba4525e506","created_at":"2026-07-21T06:36:46.384Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-1922bd1359e048a9","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.386Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-516bebee233fadba4525e506: done: 0.328419 sec","job_status":"done","queue_duration_s":0.001272,"scheduling_latency_s":0.001301,"gitaly_calls":4,"gitaly_duration_s":0.140543,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.018334,"redis_read_bytes":266,"redis_write_bytes":24004,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000898,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.00098,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.003443,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.00263,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12383,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.007291,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000163,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00039,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001641,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.014,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.130619,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.296890247,"duration_s":0.328419,"completed_at":"2026-07-21T06:36:46.716Z","load_balancing_strategy":"primary","db_duration_s":0.043645,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1835gitlab # [ 185.852589] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.717Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"07c627067809ce209faf9002","created_at":"2026-07-21T06:36:46.445Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.449Z","job_size_bytes":23,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-07c627067809ce209faf9002: start","job_status":"start","queue_duration_s":0.268451,"scheduling_latency_s":0.268467}1836gitlab # [ 185.867920] nixbot[2981]: {"ts": "2026-07-21T06:36:46.752Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/81bca3848bad4535c1e676bce30116356170a9b3 \"HTTP/1.1 201 Created\""}1837gitlab # [ 185.907291] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"07c627067809ce209faf9002","created_at":"2026-07-21T06:36:46.445Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.449Z","job_size_bytes":23,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-07c627067809ce209faf9002: done: 0.074353 sec","job_status":"done","queue_duration_s":0.268451,"scheduling_latency_s":0.268467,"redis_calls":15,"redis_duration_s":0.002004,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000197,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000334,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058774,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195555,"duration_s":0.074353,"completed_at":"2026-07-21T06:36:46.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009442,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1838gitlab # [ 185.912176] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.793Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"17236761336c549145dea145","created_at":"2026-07-21T06:36:46.465Z","trace_propagation_headers":{"sentry-trace":"fbe12aa35cc9401e9243fc6981cafecd-88bfa46cd7c64154","baggage":"sentry-trace_id=fbe12aa35cc9401e9243fc6981cafecd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P897SES05Z41ZPHZ0TRP4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.472Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-17236761336c549145dea145: start","job_status":"start","queue_duration_s":0.320498,"scheduling_latency_s":0.320513}1839gitlab # [ 185.953643] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.808Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"17236761336c549145dea145","created_at":"2026-07-21T06:36:46.465Z","trace_propagation_headers":{"sentry-trace":"fbe12aa35cc9401e9243fc6981cafecd-88bfa46cd7c64154","baggage":"sentry-trace_id=fbe12aa35cc9401e9243fc6981cafecd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P897SES05Z41ZPHZ0TRP4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.472Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-17236761336c549145dea145: done: 0.015182 sec","job_status":"done","queue_duration_s":0.320498,"scheduling_latency_s":0.320513,"redis_calls":7,"redis_duration_s":0.001,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015182,"completed_at":"2026-07-21T06:36:46.808Z","load_balancing_strategy":"primary","db_duration_s":0.001882,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1840gitlab # [ 185.965119] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.809Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2447269e01f7cc38a08e0cec","created_at":"2026-07-21T06:36:46.474Z","trace_propagation_headers":{"sentry-trace":"fbe12aa35cc9401e9243fc6981cafecd-88bfa46cd7c64154","baggage":"sentry-trace_id=fbe12aa35cc9401e9243fc6981cafecd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P897SES05Z41ZPHZ0TRP4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.477Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2447269e01f7cc38a08e0cec: start","job_status":"start","queue_duration_s":0.332124,"scheduling_latency_s":0.332138}1841gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.22 seconds)1842gitlab # [ 186.012974] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.842Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2447269e01f7cc38a08e0cec","created_at":"2026-07-21T06:36:46.474Z","trace_propagation_headers":{"sentry-trace":"fbe12aa35cc9401e9243fc6981cafecd-88bfa46cd7c64154","baggage":"sentry-trace_id=fbe12aa35cc9401e9243fc6981cafecd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P897SES05Z41ZPHZ0TRP4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.477Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2447269e01f7cc38a08e0cec: done: 0.033618 sec","job_status":"done","queue_duration_s":0.332124,"scheduling_latency_s":0.332138,"redis_calls":10,"redis_duration_s":0.0017460000000000002,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000667,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033618,"completed_at":"2026-07-21T06:36:46.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1843gitlab # [ 186.019662] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.844Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9dedcf63d5849dbd9133a7bc","created_at":"2026-07-21T06:36:46.785Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-5d964815a830436e","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:03112f69a60d00d68e8a802d87647d4b407d314e67ad5cc356dc90cb03397195","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.787Z","job_size_bytes":464,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9dedcf63d5849dbd9133a7bc: start","job_status":"start","queue_duration_s":0.056551,"scheduling_latency_s":0.056572}1844gitlab # [ 186.034347] sidekiq-cluster[2058]: {"severity":"WARN","time":"2026-07-21T06:36:46.874Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9dedcf63d5849dbd9133a7bc","created_at":"2026-07-21T06:36:46.785Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-5d964815a830436e","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:03112f69a60d00d68e8a802d87647d4b407d314e67ad5cc356dc90cb03397195","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.787Z","job_size_bytes":464,"pid":2058,"sidekiq_tid":"m2xu","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9dedcf63d5849dbd9133a7bc: fail: 0.029904 sec","job_status":"fail","queue_duration_s":0.056551,"scheduling_latency_s":0.056572,"redis_calls":7,"redis_duration_s":0.001005,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000305,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029904,"completed_at":"2026-07-21T06:36:46.873Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.003496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1845gitlab # [ 186.376654] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:46.908Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"049b12a28105cbfb0eed9d2c","created_at":"2026-07-21T06:36:46.595Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.599Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-049b12a28105cbfb0eed9d2c: start","job_status":"start","queue_duration_s":0.309207,"scheduling_latency_s":0.309237}1846gitlab # [ 186.392184] gitaly[1354]: time="2026-07-21T06:36:46.912Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P89WXEPYYB5SWSM4EQRSA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:16.711" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:46.911" grpc.time_ms=0.471 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1847gitlab # [ 186.431936] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 591848gitlab # [ 186.432700] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.002Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"049b12a28105cbfb0eed9d2c","created_at":"2026-07-21T06:36:46.595Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.599Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-049b12a28105cbfb0eed9d2c: done: 0.093956 sec","job_status":"done","queue_duration_s":0.309207,"scheduling_latency_s":0.309237,"redis_calls":10,"redis_duration_s":0.005042,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003856,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.071,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.069,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000236902,"duration_s":0.093956,"completed_at":"2026-07-21T06:36:47.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.071196,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1849gitlab # [ 186.443706] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.003Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"83cb5eaebe127075a4b80a70","created_at":"2026-07-21T06:36:46.608Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.614Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-83cb5eaebe127075a4b80a70: start","job_status":"start","queue_duration_s":0.388733,"scheduling_latency_s":0.388747}1850gitlab # [ 186.484818] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.024Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"83cb5eaebe127075a4b80a70","created_at":"2026-07-21T06:36:46.608Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.614Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-83cb5eaebe127075a4b80a70: done: 0.020936 sec","job_status":"done","queue_duration_s":0.388733,"scheduling_latency_s":0.388747,"redis_calls":10,"redis_duration_s":0.001287,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000284,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208685,"duration_s":0.020936,"completed_at":"2026-07-21T06:36:47.024Z","load_balancing_strategy":"primary","db_duration_s":0.004385,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1851gitlab # [ 186.529862] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.025Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"05e3ac6482e9316c5bd3f9a0","created_at":"2026-07-21T06:36:46.622Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.625Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-05e3ac6482e9316c5bd3f9a0: start","job_status":"start","queue_duration_s":0.399163,"scheduling_latency_s":0.399175}1852gitlab # [ 186.530608] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.042Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"05e3ac6482e9316c5bd3f9a0","created_at":"2026-07-21T06:36:46.622Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.625Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-05e3ac6482e9316c5bd3f9a0: done: 0.017748 sec","job_status":"done","queue_duration_s":0.399163,"scheduling_latency_s":0.399175,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000357,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000192762,"duration_s":0.017748,"completed_at":"2026-07-21T06:36:47.042Z","load_balancing_strategy":"primary","db_duration_s":0.002211,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1853gitlab # [ 186.548083] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.043Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"636cdb5d3a4b814cb8203486","created_at":"2026-07-21T06:36:46.626Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.630Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-636cdb5d3a4b814cb8203486: start","job_status":"start","queue_duration_s":0.413868,"scheduling_latency_s":0.413881}1854gitlab # [ 186.596707] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.059Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"636cdb5d3a4b814cb8203486","created_at":"2026-07-21T06:36:46.626Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.630Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-636cdb5d3a4b814cb8203486: done: 0.015906 sec","job_status":"done","queue_duration_s":0.413868,"scheduling_latency_s":0.413881,"redis_calls":10,"redis_duration_s":0.001248,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000262,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207009,"duration_s":0.015906,"completed_at":"2026-07-21T06:36:47.059Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002243,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1855gitlab # [ 186.600711] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.060Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f694767619b5d9fca97a054d","created_at":"2026-07-21T06:36:46.630Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.633Z","job_size_bytes":19,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f694767619b5d9fca97a054d: start","job_status":"start","queue_duration_s":0.426737,"scheduling_latency_s":0.42675}1856gitlab # [ 186.614969] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.077Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f694767619b5d9fca97a054d","created_at":"2026-07-21T06:36:46.630Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.633Z","job_size_bytes":19,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f694767619b5d9fca97a054d: done: 0.016785 sec","job_status":"done","queue_duration_s":0.426737,"scheduling_latency_s":0.42675,"redis_calls":11,"redis_duration_s":0.001367,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000244,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000184381,"duration_s":0.016785,"completed_at":"2026-07-21T06:36:47.077Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1857gitlab # [ 186.669100] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.078Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8562e0335aaaddda8b1ca7b7","created_at":"2026-07-21T06:36:46.635Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.638Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8562e0335aaaddda8b1ca7b7: start","job_status":"start","queue_duration_s":0.440492,"scheduling_latency_s":0.440525}1858gitlab # [ 186.669726] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.098Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8562e0335aaaddda8b1ca7b7","created_at":"2026-07-21T06:36:46.635Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.638Z","job_size_bytes":65,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8562e0335aaaddda8b1ca7b7: done: 0.019443 sec","job_status":"done","queue_duration_s":0.440492,"scheduling_latency_s":0.440525,"redis_calls":10,"redis_duration_s":0.001315,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000803,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000236622,"duration_s":0.019443,"completed_at":"2026-07-21T06:36:47.098Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002187,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1859gitlab # [ 186.727590] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.099Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"beec170e884c25fb49953c6b","created_at":"2026-07-21T06:36:46.639Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.640Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-beec170e884c25fb49953c6b: start","job_status":"start","queue_duration_s":0.458893,"scheduling_latency_s":0.458906}1860gitlab # [ 186.728199] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.116Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"beec170e884c25fb49953c6b","created_at":"2026-07-21T06:36:46.639Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.640Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-beec170e884c25fb49953c6b: done: 0.017185 sec","job_status":"done","queue_duration_s":0.458893,"scheduling_latency_s":0.458906,"redis_calls":7,"redis_duration_s":0.0009480000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017185,"completed_at":"2026-07-21T06:36:47.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1861gitlab # [ 186.766830] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.118Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f40a7faea14a6d81531e390f","created_at":"2026-07-21T06:36:46.656Z","trace_propagation_headers":{"sentry-trace":"fbe12aa35cc9401e9243fc6981cafecd-88bfa46cd7c64154","baggage":"sentry-trace_id=fbe12aa35cc9401e9243fc6981cafecd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P897SES05Z41ZPHZ0TRP4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.664Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f40a7faea14a6d81531e390f: start","job_status":"start","queue_duration_s":0.453964,"scheduling_latency_s":0.453977}1862gitlab # [ 186.771574] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.201Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f40a7faea14a6d81531e390f","created_at":"2026-07-21T06:36:46.656Z","trace_propagation_headers":{"sentry-trace":"fbe12aa35cc9401e9243fc6981cafecd-88bfa46cd7c64154","baggage":"sentry-trace_id=fbe12aa35cc9401e9243fc6981cafecd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P897SES05Z41ZPHZ0TRP4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.664Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f40a7faea14a6d81531e390f: done: 0.082901 sec","job_status":"done","queue_duration_s":0.453964,"scheduling_latency_s":0.453977,"gitaly_calls":3,"gitaly_duration_s":0.005547,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.005032,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000689,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001523,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000167,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000147,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000265,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059146,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.068177939,"duration_s":0.082901,"completed_at":"2026-07-21T06:36:47.201Z","load_balancing_strategy":"primary","db_duration_s":0.018259,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1863gitlab # [ 186.846085] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.202Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7f0f4f0268aaf9b1e48b22c7","created_at":"2026-07-21T06:36:46.668Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.675Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7f0f4f0268aaf9b1e48b22c7: start","job_status":"start","queue_duration_s":0.527103,"scheduling_latency_s":0.527118}1864gitlab # [ 186.846745] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.224Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7f0f4f0268aaf9b1e48b22c7","created_at":"2026-07-21T06:36:46.668Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.675Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7f0f4f0268aaf9b1e48b22c7: done: 0.022179 sec","job_status":"done","queue_duration_s":0.527103,"scheduling_latency_s":0.527118,"redis_calls":10,"redis_duration_s":0.001276,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213993,"duration_s":0.022179,"completed_at":"2026-07-21T06:36:47.224Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1865gitlab # [ 186.905887] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.225Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"60dfee7f1bef1921e99891be","created_at":"2026-07-21T06:36:46.676Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.679Z","job_size_bytes":6,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-60dfee7f1bef1921e99891be: start","job_status":"start","queue_duration_s":0.546201,"scheduling_latency_s":0.546214}1866gitlab # [ 186.906728] gitaly[1354]: time="2026-07-21T06:36:47.157Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T06:37:17.056" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:47.156" grpc.time_ms=0.559 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 186.907422] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"60dfee7f1bef1921e99891be","created_at":"2026-07-21T06:36:46.676Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.679Z","job_size_bytes":6,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-60dfee7f1bef1921e99891be: done: 0.013192 sec","job_status":"done","queue_duration_s":0.546201,"scheduling_latency_s":0.546214,"redis_calls":10,"redis_duration_s":0.0012259999999999999,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000192483,"duration_s":0.013192,"completed_at":"2026-07-21T06:36:47.238Z","load_balancing_strategy":"primary","db_duration_s":0.001231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1868gitlab # [ 186.955662] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.239Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"31a3bde9c3dac650b29787f9","created_at":"2026-07-21T06:36:46.711Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.712Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-31a3bde9c3dac650b29787f9: start","job_status":"start","queue_duration_s":0.526742,"scheduling_latency_s":0.526755}1869gitlab # [ 186.956524] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.291Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"31a3bde9c3dac650b29787f9","created_at":"2026-07-21T06:36:46.711Z","trace_propagation_headers":{"sentry-trace":"04b1b3b29f1b42dea0a8709236cc1d65-a38f9d31a0604ca0","baggage":"sentry-trace_id=04b1b3b29f1b42dea0a8709236cc1d65,sentry-release=Unknown"},"correlation_id":"01KY1P898963982KFSZ43BWHZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:46.712Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-31a3bde9c3dac650b29787f9: done: 0.051674 sec","job_status":"done","queue_duration_s":0.526742,"scheduling_latency_s":0.526755,"gitaly_calls":2,"gitaly_duration_s":0.004198,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003651,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002027,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000201,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000636,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.037426823,"duration_s":0.051674,"completed_at":"2026-07-21T06:36:47.291Z","load_balancing_strategy":"primary","db_duration_s":0.009524,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1870gitlab # [ 187.001800] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.293Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"ebe4d2af13ef9603b6b7600b","created_at":"2026-07-21T06:36:47.149Z","trace_propagation_headers":{"sentry-trace":"fbe12aa35cc9401e9243fc6981cafecd-44ce79fe99ab498a","baggage":"sentry-trace_id=fbe12aa35cc9401e9243fc6981cafecd,sentry-release=Unknown"},"correlation_id":"01KY1P897SES05Z41ZPHZ0TRP4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:47.150Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ebe4d2af13ef9603b6b7600b: start","job_status":"start","queue_duration_s":0.143165,"scheduling_latency_s":0.143191}1871gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1872gitlab # [ 187.002519] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:47.313Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"ebe4d2af13ef9603b6b7600b","created_at":"2026-07-21T06:36:47.149Z","trace_propagation_headers":{"sentry-trace":"fbe12aa35cc9401e9243fc6981cafecd-44ce79fe99ab498a","baggage":"sentry-trace_id=fbe12aa35cc9401e9243fc6981cafecd,sentry-release=Unknown"},"correlation_id":"01KY1P897SES05Z41ZPHZ0TRP4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:47.150Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ebe4d2af13ef9603b6b7600b: done: 0.020065 sec","job_status":"done","queue_duration_s":0.143165,"scheduling_latency_s":0.143191,"redis_calls":7,"redis_duration_s":0.000949,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020065,"completed_at":"2026-07-21T06:36:47.313Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1873gitlab # [ 187.053278] gitaly[1354]: time="2026-07-21T06:36:47.176Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:17.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:47.175" grpc.time_ms=0.396 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874gitlab # [ 187.053889] gitaly[1354]: time="2026-07-21T06:36:47.192Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P897SES05Z41ZPHZ0TRP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:17.091" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:47.191" grpc.time_ms=0.555 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1875gitlab # [ 187.058138] gitaly[1354]: time="2026-07-21T06:36:47.263Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T06:37:17.162" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:47.262" grpc.time_ms=0.571 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876gitlab # [ 187.098844] gitaly[1354]: time="2026-07-21T06:36:47.283Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P898963982KFSZ43BWHZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:17.182" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:47.283" grpc.time_ms=0.401 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877gitlab # [ 187.116158] gitaly[1354]: time="2026-07-21T06:36:48.018Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8AZM82AAS79HTGETYJK3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:18.017" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:48.017" grpc.time_ms=0.476 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878gitlab # [ 187.133174] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 461879gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.09 seconds)1880gitlab # [ 187.175261] nix-daemon[3278]: accepted connection from pid 3424, user nixbot1881gitlab # [ 187.176687] nix-daemon[3278]: accepted connection from pid 3425, user nixbot1882gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1883gitlab # [ 188.228433] gitaly[1354]: time="2026-07-21T06:36:49.129Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8C1KY9AKEDBBX6FD5F04 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:19.128" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:49.128" grpc.time_ms=0.802 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884gitlab # [ 188.245588] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 701885gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.12 seconds)1886gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1887gitlab # [ 189.363442] gitaly[1354]: time="2026-07-21T06:36:50.264Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8D4PC9XNBNNV3ME42YZG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:20.263" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:50.263" grpc.time_ms=0.869 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1888gitlab # [ 189.403528] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1051889gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.15 seconds)1890gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1891gitlab # [ 190.507425] gitaly[1354]: time="2026-07-21T06:36:51.408Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8E8K81PK94NKAVZDTTNM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:21.206" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:51.407" grpc.time_ms=0.846 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892gitlab # [ 190.572220] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1251893gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.17 seconds)1894gitlab # [ 191.281500] nix-daemon[3278]: accepted connection from pid 3452, user nixbot1895gitlab # [ 191.335986] gitaly[1354]: time="2026-07-21T06:36:52.237Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:22.036" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:52.236" grpc.time_ms=0.852 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1896gitlab # [ 191.389420] gitaly[1354]: time="2026-07-21T06:36:52.291Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:37:02.059" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:52.279" grpc.time_ms=11.282 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897gitlab # [ 191.462086] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.362Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7b41f2c0dfe66cf5b8d1f4ef","created_at":"2026-07-21T06:36:52.356Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-95b3e1b310d64dfc","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.360Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7b41f2c0dfe66cf5b8d1f4ef: start","job_status":"start","queue_duration_s":0.002353,"scheduling_latency_s":0.002395}1898gitlab # [ 191.511895] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee886a6781a18511736d0485295ed219b36bdc47 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 2491899gitlab # [ 191.513795] nixbot[2981]: {"ts": "2026-07-21T06:36:52.414Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee886a6781a18511736d0485295ed219b36bdc47 \"HTTP/1.1 201 Created\""}1900gitlab # [ 191.550813] gitaly[1354]: time="2026-07-21T06:36:52.451Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:52.451" grpc.time_ms=0.279 pid=1354 span.kind=server system=grpc1901gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1902gitlab # [ 191.581364] gitaly[1354]: time="2026-07-21T06:36:52.483Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8FAAHMF3KJ0PQC0F9N8V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:22.282" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:52.482" grpc.time_ms=0.388 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1903gitlab # [ 191.609482] gitaly[1354]: time="2026-07-21T06:36:52.511Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:52.511" grpc.time_ms=0.329 pid=1354 span.kind=server system=grpc1904gitlab # [ 191.624034] gitaly[1354]: time="2026-07-21T06:36:52.525Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1P8FAAHMF3KJ0PQC0F9N8V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:37:02.300" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:52.520" grpc.time_ms=5.418 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 191.641602] gitaly[1354]: time="2026-07-21T06:36:52.542Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8FCN3D4PEQK6JPS0SM5J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:22.541" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:52.541" grpc.time_ms=0.349 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1906gitlab # [ 191.641897] gitaly[1354]: time="2026-07-21T06:36:52.542Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:52.542" grpc.time_ms=0.194 pid=1354 span.kind=server system=grpc1907gitlab # [ 191.654275] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 671 "" "curl/8.20.0" 531908gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.08 seconds)1909gitlab # [ 191.673797] gitaly[1354]: time="2026-07-21T06:36:52.575Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 diskcache=66ef852f-d4ec-44ea-8d54-c168461e67d2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:22.326" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:52.426" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910gitlab # [ 191.673979] gitaly[1354]: time="2026-07-21T06:36:52.575Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=127 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7215 command.oublock=0 command.real_time_ms=139 command.spawn_token_fork_ms=0 command.system_time_ms=94 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:22.326" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:52.426" grpc.time_ms=149.184 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911gitlab # [ 191.696069] nixbot[2981]: {"ts": "2026-07-21T06:36:52.597Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee886a6781a18511736d0485295ed219b36bdc47 \"HTTP/1.1 201 Created\""}1912gitlab # [ 191.698223] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee886a6781a18511736d0485295ed219b36bdc47 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1731913gitlab # [ 191.708265] gitaly[1354]: time="2026-07-21T06:36:52.610Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T06:37:22.509" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:52.609" grpc.time_ms=0.598 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab # [ 191.736135] gitaly[1354]: time="2026-07-21T06:36:52.638Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:22.537" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:52.637" grpc.time_ms=0.428 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab # [ 191.784161] gitaly[1354]: time="2026-07-21T06:36:52.685Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:22.584" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:52.685" grpc.time_ms=0.835 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916gitlab # [ 191.790590] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.692Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"0244146c2fc94206dc3e78e8","created_at":"2026-07-21T06:36:52.690Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"7b41f2c0dfe66cf5b8d1f4ef","job_size_bytes":3,"pid":2058,"job_status":"deduplicated","message":"PipelineProcessWorker JID-0244146c2fc94206dc3e78e8: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1917gitlab # [ 191.791693] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.693Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7b41f2c0dfe66cf5b8d1f4ef","created_at":"2026-07-21T06:36:52.356Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-95b3e1b310d64dfc","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.360Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2058,"job_status":"rescheduled","message":"PipelineProcessWorker JID-7b41f2c0dfe66cf5b8d1f4ef: rescheduled"}1918gitlab # [ 191.796302] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.697Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7b41f2c0dfe66cf5b8d1f4ef","created_at":"2026-07-21T06:36:52.356Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-95b3e1b310d64dfc","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.360Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7b41f2c0dfe66cf5b8d1f4ef: done: 0.335243 sec","job_status":"done","queue_duration_s":0.002353,"scheduling_latency_s":0.002395,"gitaly_calls":4,"gitaly_duration_s":0.180584,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.016264,"redis_read_bytes":262,"redis_write_bytes":23389,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001003,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002221,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001067,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.003432,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14470,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.005356,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000213,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000515,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001454,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.111454,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.310505106,"duration_s":0.335243,"completed_at":"2026-07-21T06:36:52.697Z","load_balancing_strategy":"primary","db_duration_s":0.034081,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1919gitlab # [ 191.839788] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.699Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"6b76cd8bc8113042ef1dccaf","created_at":"2026-07-21T06:36:52.415Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.419Z","job_size_bytes":23,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-6b76cd8bc8113042ef1dccaf: start","job_status":"start","queue_duration_s":0.279813,"scheduling_latency_s":0.279828}1920gitlab # [ 191.887713] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.724Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"6b76cd8bc8113042ef1dccaf","created_at":"2026-07-21T06:36:52.415Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.419Z","job_size_bytes":23,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-6b76cd8bc8113042ef1dccaf: done: 0.025134 sec","job_status":"done","queue_duration_s":0.279813,"scheduling_latency_s":0.279828,"redis_calls":7,"redis_duration_s":0.001679,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025134,"completed_at":"2026-07-21T06:36:52.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006959,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1921gitlab # [ 191.893806] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.725Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6553543c41c961374b40b4a2","created_at":"2026-07-21T06:36:52.580Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.582Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6553543c41c961374b40b4a2: start","job_status":"start","queue_duration_s":0.142646,"scheduling_latency_s":0.142661}1922gitlab # [ 191.944625] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.740Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6553543c41c961374b40b4a2","created_at":"2026-07-21T06:36:52.580Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.582Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6553543c41c961374b40b4a2: done: 0.014783 sec","job_status":"done","queue_duration_s":0.142646,"scheduling_latency_s":0.142661,"redis_calls":7,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001061,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014783,"completed_at":"2026-07-21T06:36:52.740Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1923gitlab # [ 191.948020] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.741Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6bef4f33b4cdea5f373fd4f4","created_at":"2026-07-21T06:36:52.588Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.590Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6bef4f33b4cdea5f373fd4f4: start","job_status":"start","queue_duration_s":0.150918,"scheduling_latency_s":0.150931}1924gitlab # [ 191.988549] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.761Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6bef4f33b4cdea5f373fd4f4","created_at":"2026-07-21T06:36:52.588Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.590Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6bef4f33b4cdea5f373fd4f4: done: 0.020023 sec","job_status":"done","queue_duration_s":0.150918,"scheduling_latency_s":0.150931,"redis_calls":7,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001148,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020023,"completed_at":"2026-07-21T06:36:52.761Z","load_balancing_strategy":"primary","db_duration_s":0.007186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1925gitlab # [ 192.000542] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.762Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"fb8413d3cfdbc2c32e46c807","created_at":"2026-07-21T06:36:52.595Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.596Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fb8413d3cfdbc2c32e46c807: start","job_status":"start","queue_duration_s":0.166028,"scheduling_latency_s":0.166043}1926gitlab # [ 192.042445] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.780Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"fb8413d3cfdbc2c32e46c807","created_at":"2026-07-21T06:36:52.595Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.596Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fb8413d3cfdbc2c32e46c807: done: 0.017908 sec","job_status":"done","queue_duration_s":0.166028,"scheduling_latency_s":0.166043,"redis_calls":10,"redis_duration_s":0.002229,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000613,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017908,"completed_at":"2026-07-21T06:36:52.780Z","load_balancing_strategy":"primary","db_duration_s":0.004094,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1927gitlab # [ 192.084786] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.781Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ac3dd1adcda14e084cf4b6cc","created_at":"2026-07-21T06:36:52.596Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.598Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ac3dd1adcda14e084cf4b6cc: start","job_status":"start","queue_duration_s":0.182856,"scheduling_latency_s":0.182869}1928gitlab # [ 192.085802] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.796Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ac3dd1adcda14e084cf4b6cc","created_at":"2026-07-21T06:36:52.596Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.598Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ac3dd1adcda14e084cf4b6cc: done: 0.015353 sec","job_status":"done","queue_duration_s":0.182856,"scheduling_latency_s":0.182869,"redis_calls":7,"redis_duration_s":0.0016879999999999998,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001179,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000509,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015353,"completed_at":"2026-07-21T06:36:52.796Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004563,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1929gitlab # [ 192.127166] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.797Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"580d7420b9b19bfca4d701a9","created_at":"2026-07-21T06:36:52.599Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.601Z","job_size_bytes":19,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-580d7420b9b19bfca4d701a9: start","job_status":"start","queue_duration_s":0.196511,"scheduling_latency_s":0.196526}1930gitlab # [ 192.127887] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.813Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"580d7420b9b19bfca4d701a9","created_at":"2026-07-21T06:36:52.599Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.601Z","job_size_bytes":19,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-580d7420b9b19bfca4d701a9: done: 0.015541 sec","job_status":"done","queue_duration_s":0.196511,"scheduling_latency_s":0.196526,"redis_calls":8,"redis_duration_s":0.001711,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001276,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015541,"completed_at":"2026-07-21T06:36:52.813Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003606,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1931gitlab # [ 192.172630] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.814Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"557ee8c82a4bf9c1b672ad6d","created_at":"2026-07-21T06:36:52.602Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.603Z","job_size_bytes":66,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-557ee8c82a4bf9c1b672ad6d: start","job_status":"start","queue_duration_s":0.211358,"scheduling_latency_s":0.211372}1932gitlab # [ 192.176939] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.825Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"557ee8c82a4bf9c1b672ad6d","created_at":"2026-07-21T06:36:52.602Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.603Z","job_size_bytes":66,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-557ee8c82a4bf9c1b672ad6d: done: 0.011426 sec","job_status":"done","queue_duration_s":0.211358,"scheduling_latency_s":0.211372,"redis_calls":7,"redis_duration_s":0.001531,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011426,"completed_at":"2026-07-21T06:36:52.825Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1933gitlab # [ 192.219844] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.826Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6e98c9c174e71835b58f48ce","created_at":"2026-07-21T06:36:52.603Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.604Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6e98c9c174e71835b58f48ce: start","job_status":"start","queue_duration_s":0.22212,"scheduling_latency_s":0.222133}1934gitlab # [ 192.232337] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.851Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"6e98c9c174e71835b58f48ce","created_at":"2026-07-21T06:36:52.603Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.604Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-6e98c9c174e71835b58f48ce: done: 0.024923 sec","job_status":"done","queue_duration_s":0.22212,"scheduling_latency_s":0.222133,"redis_calls":7,"redis_duration_s":0.001956,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001387,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000569,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024923,"completed_at":"2026-07-21T06:36:52.851Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003922,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1935gitlab # [ 192.273772] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.854Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"37e4140251e2b29b5fa2eaa2","created_at":"2026-07-21T06:36:52.656Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.658Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-37e4140251e2b29b5fa2eaa2: start","job_status":"start","queue_duration_s":0.195749,"scheduling_latency_s":0.19579}1936gitlab # [ 192.318320] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.885Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"37e4140251e2b29b5fa2eaa2","created_at":"2026-07-21T06:36:52.656Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.658Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-37e4140251e2b29b5fa2eaa2: done: 0.031269 sec","job_status":"done","queue_duration_s":0.195749,"scheduling_latency_s":0.19579,"redis_calls":7,"redis_duration_s":0.002264,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001618,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000646,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031269,"completed_at":"2026-07-21T06:36:52.885Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004746,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1937gitlab # [ 192.360195] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.888Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"72fdb01f79300daf5c864368","created_at":"2026-07-21T06:36:52.659Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.660Z","job_size_bytes":6,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-72fdb01f79300daf5c864368: start","job_status":"start","queue_duration_s":0.227138,"scheduling_latency_s":0.227173}1938gitlab # [ 192.361456] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.905Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"72fdb01f79300daf5c864368","created_at":"2026-07-21T06:36:52.659Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.660Z","job_size_bytes":6,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-72fdb01f79300daf5c864368: done: 0.01714 sec","job_status":"done","queue_duration_s":0.227138,"scheduling_latency_s":0.227173,"redis_calls":7,"redis_duration_s":0.001852,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001344,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01714,"completed_at":"2026-07-21T06:36:52.905Z","load_balancing_strategy":"primary","db_duration_s":0.00055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1939gitlab # [ 192.402187] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.906Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"f828314d5cc8b9dafe08a321","created_at":"2026-07-21T06:36:52.661Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.665Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f828314d5cc8b9dafe08a321: start","job_status":"start","queue_duration_s":0.241582,"scheduling_latency_s":0.241607}1940gitlab # [ 192.403281] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.928Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"f828314d5cc8b9dafe08a321","created_at":"2026-07-21T06:36:52.661Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.665Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f828314d5cc8b9dafe08a321: done: 0.021478 sec","job_status":"done","queue_duration_s":0.241582,"scheduling_latency_s":0.241607,"redis_calls":10,"redis_duration_s":0.002108,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001285,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000281041,"duration_s":0.021478,"completed_at":"2026-07-21T06:36:52.928Z","load_balancing_strategy":"primary","db_duration_s":0.004549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1941gitlab # [ 192.433849] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:52.929Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"bd7cf65ae3be663f2bc1c284","created_at":"2026-07-21T06:36:52.693Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.694Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bd7cf65ae3be663f2bc1c284: start","job_status":"start","queue_duration_s":0.234771,"scheduling_latency_s":0.234784}1942gitlab # [ 192.485226] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:53.038Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"bd7cf65ae3be663f2bc1c284","created_at":"2026-07-21T06:36:52.693Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-6d0d2c15cddc494a","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.694Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bd7cf65ae3be663f2bc1c284: done: 0.10924 sec","job_status":"done","queue_duration_s":0.234771,"scheduling_latency_s":0.234784,"gitaly_calls":3,"gitaly_duration_s":0.008213,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007664,"redis_read_bytes":98,"redis_write_bytes":6012,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000855,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001975,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000281,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1131,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002054,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000246,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000432,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064914,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.085202728,"duration_s":0.10924,"completed_at":"2026-07-21T06:36:53.038Z","load_balancing_strategy":"primary","db_duration_s":0.03377,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1943gitlab # [ 192.496929] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:53.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b9ddd1bb426ddba833e8df1f","created_at":"2026-07-21T06:36:52.967Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-84e974fff0cb4a91","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.968Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b9ddd1bb426ddba833e8df1f: start","job_status":"start","queue_duration_s":0.072044,"scheduling_latency_s":0.072077}1944gitlab # [ 192.512807] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:53.074Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b9ddd1bb426ddba833e8df1f","created_at":"2026-07-21T06:36:52.967Z","trace_propagation_headers":{"sentry-trace":"22318369c5814aa797c326c2e7a7ac7b-84e974fff0cb4a91","baggage":"sentry-trace_id=22318369c5814aa797c326c2e7a7ac7b,sentry-release=Unknown"},"correlation_id":"01KY1P8F23MPDGS3J04TJXK3Z6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:52.968Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b9ddd1bb426ddba833e8df1f: done: 0.033823 sec","job_status":"done","queue_duration_s":0.072044,"scheduling_latency_s":0.072077,"redis_calls":7,"redis_duration_s":0.002099,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001505,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000594,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033823,"completed_at":"2026-07-21T06:36:53.074Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004001,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1945gitlab # [ 192.585466] gitaly[1354]: time="2026-07-21T06:36:52.977Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T06:37:22.875" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:52.975" grpc.time_ms=1.232 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1946gitlab # [ 192.586221] gitaly[1354]: time="2026-07-21T06:36:52.999Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:22.897" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:52.998" grpc.time_ms=0.843 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1947gitlab # [ 192.586846] gitaly[1354]: time="2026-07-21T06:36:53.020Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8F23MPDGS3J04TJXK3Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:23.018" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:53.019" grpc.time_ms=0.859 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1948gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1949gitlab # [ 192.775866] gitaly[1354]: time="2026-07-21T06:36:53.677Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8GFKKCMKTV4DQS47GCW9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:23.675" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:53.676" grpc.time_ms=0.99 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1950gitlab # [ 192.811928] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 931951gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.15 seconds)1952gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1953gitlab # [ 193.912832] gitaly[1354]: time="2026-07-21T06:36:54.813Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8HK372Y8S4FYJTZH6VH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:24.812" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:54.812" grpc.time_ms=0.869 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1954gitlab # [ 193.970196] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1151955gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.16 seconds)1956gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1957gitlab # [ 195.068062] gitaly[1354]: time="2026-07-21T06:36:55.969Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8JQAQZT6299SCT729GJJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:25.767" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:55.968" grpc.time_ms=0.8 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1958gitlab # [ 195.131831] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1171959gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.16 seconds)1960gitlab # [ 195.639539] nix-daemon[3278]: accepted connection from pid 3529, user nixbot1961gitlab # [ 195.729959] gitaly[1354]: time="2026-07-21T06:36:56.631Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:26.630" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:56.630" grpc.time_ms=0.908 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962gitlab # [ 195.764243] gitaly[1354]: time="2026-07-21T06:36:56.666Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T06:37:06.223" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:56.653" grpc.time_ms=12.031 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 195.812319] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:56.712Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ab7c9131a26ff3b86d9f0234","created_at":"2026-07-21T06:36:56.706Z","trace_propagation_headers":{"sentry-trace":"08763563d93749589d4755f64fbdde84-82e5990b904647b1","baggage":"sentry-trace_id=08763563d93749589d4755f64fbdde84,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P8KBS9HVPXT6TX83HZ5TP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:56.710Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab7c9131a26ff3b86d9f0234: start","job_status":"start","queue_duration_s":0.001958,"scheduling_latency_s":0.002005}1964gitlab # [ 195.842762] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee886a6781a18511736d0485295ed219b36bdc47 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1631965gitlab # [ 195.843815] nixbot[2981]: {"ts": "2026-07-21T06:36:56.734Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee886a6781a18511736d0485295ed219b36bdc47 \"HTTP/1.1 201 Created\""}1966gitlab # [ 195.947562] gitaly[1354]: time="2026-07-21T06:36:56.849Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:56.848" grpc.time_ms=0.487 pid=1354 span.kind=server system=grpc1967gitlab # [ 195.957913] gitaly[1354]: time="2026-07-21T06:36:56.852Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P8KJTX95G3APD423YF9JP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:56.842" grpc.time_ms=8.719 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1968gitlab # [ 195.970841] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:56 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 571969gitlab # [ 195.993391] gitaly[1354]: time="2026-07-21T06:36:56.894Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:56.894" grpc.time_ms=0.288 pid=1354 span.kind=server system=grpc1970gitlab # [ 196.004832] nix-daemon[3278]: accepted connection from pid 3560, user nixbot1971gitlab # [ 196.006864] gitaly[1354]: time="2026-07-21T06:36:56.908Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KMN7H1KTS5TSNMV79SH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:56.891" pid=1354 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1972gitlab # [ 196.007231] gitaly[1354]: time="2026-07-21T06:36:56.908Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KMN7H1KTS5TSNMV79SH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:56.891" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1973gitlab # [ 196.009339] gitlab-workhorse[1292]: time="2026-07-21T06:36:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671974gitlab # [ 196.009934] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 561975gitlab # [ 196.010595] gitaly[1354]: time="2026-07-21T06:36:56.910Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KMN7H1KTS5TSNMV79SH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T06:36:56.891" grpc.time_ms=18.47 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc1976gitlab # [ 196.063498] gitaly[1354]: time="2026-07-21T06:36:56.965Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T06:36:56.964" grpc.time_ms=0.334 pid=1354 span.kind=server system=grpc1977gitlab # [ 196.075192] gitaly[1354]: time="2026-07-21T06:36:56.975Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:26.712" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:56.813" operations="[repack_refs]" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1978gitlab # [ 196.084176] gitaly[1354]: time="2026-07-21T06:36:56.986Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:26.712" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:56.813" pid=1354 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-07-21 06:36:38.355689739 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root1979gitlab # [ 196.100053] gitaly[1354]: time="2026-07-21T06:36:57.002Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP diskcache=5cf73ad1-f70b-4b87-898f-537250a445c1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:26.712" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:56.813" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980gitlab # [ 196.107056] gitaly[1354]: time="2026-07-21T06:36:57.008Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:26.712" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:56.813" pid=1354 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root1981gitlab # [ 196.107772] gitaly[1354]: time="2026-07-21T06:36:57.009Z" level=info msg="optimized repository" clean_stale_data_time=22.306187ms clean_work_trees_time="78.222µs" commit_graph_time="3.632µs" correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.911µs" pid=1354 prune_time="3.353µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="3.911µs" storage_name=default system=housekeeping total_time=22.490289ms1982gitlab # [ 196.108354] gitaly[1354]: time="2026-07-21T06:36:57.010Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:26.712" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:56.813" pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses1984gitlab # [ 196.133423] gitaly[1354]: time="2026-07-21T06:36:57.035Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=80 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7408 command.oublock=0 command.real_time_ms=162 command.spawn_token_fork_ms=1 command.system_time_ms=55 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T06:37:26.712" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T06:36:56.813" grpc.time_ms=189.06 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985gitlab # [ 196.177186] gitaly[1354]: time="2026-07-21T06:36:57.079Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T06:37:27.078" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:57.078" grpc.time_ms=0.687 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986gitlab # [ 196.195121] gitaly[1354]: time="2026-07-21T06:36:57.096Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KV6KR691S3YXVMT8CMC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:27.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:57.096" grpc.time_ms=0.369 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987gitlab # [ 196.206834] gitaly[1354]: time="2026-07-21T06:36:57.099Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:27.098" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:57.098" grpc.time_ms=0.61 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988gitlab # [ 196.218144] nix-daemon[3278]: accepted connection from pid 3609, user nixbot1989gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses, in 0.08 seconds)1990gitlab # [ 196.219523] gitlab-workhorse[1292]: gitlab 2001:db8:1::1 - - [2026/07/21:06:36:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee886a6781a18511736d0485295ed219b36bdc47/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 551991(finished: subtest: eval and build post commit statuses, in 10.39 seconds)1992(finished: run the VM test script, in 244.25 seconds)1993gitlab # [ 196.238766] gitaly[1354]: time="2026-07-21T06:36:57.140Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1P8KBS9HVPXT6TX83HZ5TP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T06:37:27.038" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T06:36:57.139" grpc.time_ms=0.795 pid=1354 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1994gitlab # [ 196.248200] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:57.149Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ab7c9131a26ff3b86d9f0234","created_at":"2026-07-21T06:36:56.706Z","trace_propagation_headers":{"sentry-trace":"08763563d93749589d4755f64fbdde84-82e5990b904647b1","baggage":"sentry-trace_id=08763563d93749589d4755f64fbdde84,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1P8KBS9HVPXT6TX83HZ5TP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:56.710Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab7c9131a26ff3b86d9f0234: done: 0.437809 sec","job_status":"done","queue_duration_s":0.001958,"scheduling_latency_s":0.002005,"gitaly_calls":4,"gitaly_duration_s":0.232233,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.02083,"redis_read_bytes":173,"redis_write_bytes":20926,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001583,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002902,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.003309,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13379,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.009186,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000331,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000338,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001598,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.129958,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.410696052,"duration_s":0.437809,"completed_at":"2026-07-21T06:36:57.149Z","load_balancing_strategy":"primary","db_duration_s":0.056024,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1995gitlab # [ 196.256752] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:57.151Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d72c12ca84548a75d2903112","created_at":"2026-07-21T06:36:56.803Z","trace_propagation_headers":{"sentry-trace":"08763563d93749589d4755f64fbdde84-bf4bd76a4c4d48a8","baggage":"sentry-trace_id=08763563d93749589d4755f64fbdde84,sentry-release=Unknown"},"correlation_id":"01KY1P8KBS9HVPXT6TX83HZ5TP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:56.806Z","job_size_bytes":23,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d72c12ca84548a75d2903112: start","job_status":"start","queue_duration_s":0.345405,"scheduling_latency_s":0.345423}1996gitlab # [ 196.273538] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:57.175Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d72c12ca84548a75d2903112","created_at":"2026-07-21T06:36:56.803Z","trace_propagation_headers":{"sentry-trace":"08763563d93749589d4755f64fbdde84-bf4bd76a4c4d48a8","baggage":"sentry-trace_id=08763563d93749589d4755f64fbdde84,sentry-release=Unknown"},"correlation_id":"01KY1P8KBS9HVPXT6TX83HZ5TP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:56.806Z","job_size_bytes":23,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d72c12ca84548a75d2903112: done: 0.023402 sec","job_status":"done","queue_duration_s":0.345405,"scheduling_latency_s":0.345423,"redis_calls":7,"redis_duration_s":0.00102,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000698,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023402,"completed_at":"2026-07-21T06:36:57.175Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003895,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1997gitlab # [ 196.312271] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:57.176Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"103fdb91b21a3bfe45fd190b","created_at":"2026-07-21T06:36:57.039Z","trace_propagation_headers":{"sentry-trace":"08763563d93749589d4755f64fbdde84-bf4bd76a4c4d48a8","baggage":"sentry-trace_id=08763563d93749589d4755f64fbdde84,sentry-release=Unknown"},"correlation_id":"01KY1P8KBS9HVPXT6TX83HZ5TP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:57.047Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-103fdb91b21a3bfe45fd190b: start","job_status":"start","queue_duration_s":0.129148,"scheduling_latency_s":0.129164}1998test script finished in 244.43s1999gitlab # [ 196.360648] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:57.189Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"103fdb91b21a3bfe45fd190b","created_at":"2026-07-21T06:36:57.039Z","trace_propagation_headers":{"sentry-trace":"08763563d93749589d4755f64fbdde84-bf4bd76a4c4d48a8","baggage":"sentry-trace_id=08763563d93749589d4755f64fbdde84,sentry-release=Unknown"},"correlation_id":"01KY1P8KBS9HVPXT6TX83HZ5TP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-21T06:36:57.047Z","job_size_bytes":3,"pid":2058,"sidekiq_tid":"m5n6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-103fdb91b21a3bfe45fd190b: done: 0.013097 sec","job_status":"done","queue_duration_s":0.129148,"scheduling_latency_s":0.129164,"redis_calls":7,"redis_duration_s":0.000938,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008237,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013097,"completed_at":"2026-07-21T06:36:57.189Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2000cleanup2001kill QemuMachine (pid 13)2002gitlab # [ 196.363711] sidekiq-cluster[2058]: {"severity":"INFO","time":"2026-07-21T06:36:57.190Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"83902a2d176ce8cafcbda8c6","created_at":"2026-07-21T06:36:57.055Z","trace_propagation_headers":{"sentry-trace":"08763563d93749589d4755f64fbdde84-bf4bd76a4c4d48a8","baggage":"sentry-trace_id=08763563d93749589d4755f64fbdde84,sentry-release=Unknown"},"correlation_id":"01KY1P8KBS9HVPXT6TX83HZ5TP",qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2003gitlab # "2004(finished: cleanup, in 1.42 seconds)2005additionally exposed symbols:2006 gitlab,2007 vlan1,2008 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2009[]2010[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-21T06:36:42.870Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2011[]2012[]2013[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-21T06:36:46.191823Z', 'started_at': None, 'finished_at': None}]2014[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2015[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2016[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2017[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2018[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2019[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': '2026-07-21T06:36:52.345Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2020[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': '2026-07-21T06:36:52.345Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-21T06:36:52.578Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2021[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': '2026-07-21T06:36:52.345Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-21T06:36:52.578Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2022[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': '2026-07-21T06:36:52.345Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-21T06:36:52.578Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2023[{'id': 2, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T06:36:46.647Z', 'started_at': None, 'finished_at': '2026-07-21T06:36:52.345Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ee886a6781a18511736d0485295ed219b36bdc47', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-21T06:36:52.578Z', 'started_at': None, 'finished_at': '2026-07-21T06:36:56.694Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]